redis-benchmarks-specification

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

Tooling changes: Setup agnostic cli benchmark runner (solely client)

• Added --client_aggregated_results_folder and --preserve_temporary_client_dirs to redis-benchmarks-spec-client-runner @filipecosta90 (#95)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

• Extended benchmark with 9 SET datatype new benchmark @filipecosta90 (#94)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

• Added 11 extra STRING benchmarks @filipecosta90 (#93)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

• Added extra LRANGE, LPUSH, LPOP and RPOP benchmark variations (10B, 100B, and 1KiB) @filipecosta90 (#91)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

Tooling changes: Self-contained coordinator

• Enabled CPU pinning on self-contained-coordinator @filipecosta90 (#90)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

Tooling changes: Benchmark artifacts builder

• Enable profilers on self-contained-coordinator and bumped dependencies to the latest version @filipecosta90 (#89)

Tooling changes: Self-contained coordinator

• Enable profilers on self-contained-coordinator and bumped dependencies to the latest version @filipecosta90 (#89)
• Included XREAD, XADD, and ZRANGE benchmarks @filipecosta90 (#88)

Tooling changes: Setup agnostic cli benchmark runner (solely client)

• Included XREAD, XADD, and ZRANGE benchmarks @filipecosta90 (#88)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

• Added memtier_benchmark-1key-zset-1M-elements-zrevrange-5-elements. @filipecosta90 (#83)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

• Fixed multiple TLS tests fail on 2nd test due to tls cert/key filenames being replaced @filipecosta90 (#79)

Tooling changes: Self-contained coordinator

• In case of unexpected failure on process_self_contained_coordinator_stream we log it and recover @filipecosta90 (#82)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

Tooling changes: Setup agnostic cli benchmark runner (solely client)

• Enable TLS benchmarks on client-runner and memtier_benchmark @filipecosta90 (#77)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.
• *Spec tool implementation: Setup agnostic cli benchmark runner (solely client): contains the client utility that triggers the actual benchmarks against an endpoint provided. This tool is setup agnostic and expects the DB to be properly spinned beforehand.

Release changes:

• Added doc reference to cli benchmark runner @filipecosta90 (#76)
• Fixed pipeline 10 HSET benchmarks @filipecosta90 (#74)
• Added HSET tests (with/without pipeline) @filipecosta90 (#73)

Tooling changes: Setup agnostic cli benchmark runner (solely client)

• Extended benchmarks with LPOP, RPOP, LPUSH, HGETALL, HGET, HVALS, HKE… @filipecosta90 (#75)
• Allow to specify a test to run on cli runner; Included pipeline 10 redis-benchmark tests @filipecosta90 (#72)
• Added cpuset start position for client runner @filipecosta90 (#71)
• Added setup agnostic cli benchmark runner (solely client): redis-benchmarks-spec-client-runner @filipecosta90 (#70)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Release changes:

• Introduced redis-benchmark-full-suite-1Mkeys-1KiB benchmark variation @filipecosta90 (#68)
• Extended logging for action:trigger-benchmark @filipecosta90 (#65)

Tooling changes: Self-contained coordinator

• Fixed redis-benchmarks-spec-api response_data being used before assignement @filipecosta90 (#48)
• [add] Enabled the build-variants property on test suite spec (to avoid exploding total variants per build) @filipecosta90 (#67)
• memtier_benchmark support + data preload @filipecosta90 (#66)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Release changes:

• Included Issue Bug Report GH template @filipecosta90 (#61)

Tooling changes: Benchmark artifacts builder

• Fixed wrong redis dependencies building due to hdr_histogram not existing on < 6.2.X @filipecosta90 (#58)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Release changes:

Tooling changes: Benchmark artifacts builder

• Enabled git version benchmark builds/runs @filipecosta90 (#54)

Tooling changes: Self-contained coordinator

• Enabled git version benchmark builds/runs @filipecosta90 (#54)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Release changes:

Tooling changes: Self-contained coordinator

• [fix] Fixed redistimeseries.client error o self-contained-coordinator @filipecosta90 (#53)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Release changes:

• Updated docs to include how-to add a new benchmark platform @filipecosta90 (#51)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Release changes:

Tooling changes: API

• [add] Using strings to store the binaries produced by tooling @filipecosta90 (#50)

Tooling changes: Benchmark artifacts builder

• [add] Using strings to store the binaries produced by tooling @filipecosta90 (#50)

Tooling changes: Self-contained coordinator

• [add] Using strings to store the binaries produced by tooling @filipecosta90 (#50)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Tooling changes: API

• Moved from gh action API to gh webhook API @filipecosta90 (#49)
• [fix] Fixed response_data being used before assignement @filipecosta90 (#45)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Release changes:

Tooling changes: Benchmark artifacts builder

• [fix] Fixed docker images unnecessary prefetching from hub @filipecosta90 (#44)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Release changes:

Tooling changes: API

• [add] Introduce health_check_interval, socket_connect_timeout, and socket_keepalive settings to Redis connections @filipecosta90 (#40)

Tooling changes: Benchmark artifacts builder

• [add] Introduce health_check_interval, socket_connect_timeout, and socket_keepalive settings to Redis connections @filipecosta90 (#40)

Tooling changes: Self-contained coordinator

• [add] Introduce health_check_interval, socket_connect_timeout, and socket_keepalive settings to Redis connections @filipecosta90 (#40)

Redis performance and observability spec change

Project status:

The following is a high level status report for currently available specs and tools.

• Benchmark platforms specification: Experimental components are released and available for beta testing.
• Benchmark topologies specification: Experimental components are released and available for beta testing.
• Benchmark artifacts builder specification: Experimental components are released and available for beta testing.
• Test-suites specification: Experimental components are released and available for beta testing.
• Spec tool implementation: API: Experimental components are released and available for beta testing.
• Spec tool implementation: Benchmark artifacts builder: Experimental components are released and available for beta testing.
• Spec tool implementation: Self-contained coordinator: Experimental components are released and available for beta testing.
• Spec tool implementation: Large scale coordinator: Draft components are under design, and have not been added to the specification.

Release changes:

Tooling changes: API

• [add] Added redis-benchmarks-spec-cli trigger tool; Enabling historical data benchmarks triggering @filipecosta90 (#38)

Tooling changes: Benchmark artifacts builder

• [add] Added redis-benchmarks-spec-cli trigger tool; Enabling historical data benchmarks triggering @filipecosta90 (#38)

Tooling changes: Self-contained coordinator

• [add] Added redis-benchmarks-spec-cli trigger tool; Enabling historical data benchmarks triggering @filipecosta90 (#38)