redis/redis-benchmarks-specification

View on GitHub

The Redis benchmarks specification describes the cross-language/tools requirements and expectations to foster performance and observability standards around redis related technologies. Members from both industry and academia, including organizations and individuals are encouraged to contribute.

Stars: 42
Forks: 16
Open beginner issues: 0
Indexed issues: 77
Avg merge: 18h 20m
Merged PRs (30d): 21

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 11, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted

Beginner friendly Everything

Open All

77 open issues indexed

Blocking-command specs export no throughput/latency metric: playbook specs run with 0 Ops/sec series, memtier blocking specs never run Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

redis/redis-benchmarks-specification#572 · Opened Sep 19, 2026
Add an L3-resident io-threads string spec (all 35 io-threads specs are >=2.5x L3) Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

redis/redis-benchmarks-specification#571 · Opened Sep 18, 2026
Coverage: no benchmark spec for ZRANDMEMBER Open

Difficulty 3/5 1-2 days Newbie friendliness 78/100

redis/redis-benchmarks-specification#568 · Opened Sep 16, 2026
Add benchmark coverage for AOF load and cold-page-cache RDB load Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

redis/redis-benchmarks-specification#567 · Opened Sep 15, 2026
Add benchmark coverage for replica-side replication stream apply (io-threads master link) Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

redis/redis-benchmarks-specification#565 · Opened Sep 12, 2026
Add benchmark coverage for LZF decompression (RDB load and RESTORE paths) Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

redis/redis-benchmarks-specification#559 · Opened Sep 10, 2026
coverage gap: GEO*STORE variants (GEORADIUS_STORE / GEORADIUSBYMEMBER_STORE / GEOSEARCHSTORE) Open

Difficulty 2/5 1-3 hours Newbie friendliness 82/100

redis/redis-benchmarks-specification#555 · Opened Sep 6, 2026
dbconfig.check.keyspacelen is declared by 458 specs but enforced by none on the live coordinator path Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

redis/redis-benchmarks-specification#553 · Opened Sep 2, 2026
BGSAVE-duration spec: wait_for_bgsave block could start right after the client run instead of after the full post-run tail Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

redis/redis-benchmarks-specification#552 · 1 comment · Opened Sep 2, 2026
Several continue-past-teardown paths leak the DB container in process_self_contained_coordinator_stream Open

Difficulty 4/5 3-5 days Newbie friendliness 58/100

redis/redis-benchmarks-specification#551 · 1 comment · Opened Sep 2, 2026
merge_default_and_config_metrics() mutates the shared default_metrics list in place across a coordinator run Open

Difficulty 2/5 1-3 hours Newbie friendliness 88/100

redis/redis-benchmarks-specification#550 · 1 comment · Opened Sep 2, 2026
rdb.c is unmapped in files-to-groups.json, so BGSAVE-duration specs aren't webhook-reached by persistence PRs Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

redis/redis-benchmarks-specification#549 · 3 comments · Opened Sep 2, 2026
Add ZRANGEBYLEX / ZREVRANGEBYLEX / ZLEXCOUNT / ZRANGESTORE BYLEX coverage Open

Difficulty 3/5 1-2 days Newbie friendliness 78/100

redis/redis-benchmarks-specification#548 · Opened Sep 2, 2026
[SPEC]: remaining blocking commands with zero coverage after #442/#504 — BLMOVE, BRPOPLPUSH, BZPOPMAX, BLMPOP, BZMPOP, WAIT, WAITAOF, XREAD BLOCK Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

redis/redis-benchmarks-specification#543 · Opened Aug 31, 2026
[BUG]: commands.json is 5 minor releases behind and disagrees with the suites it validates Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

redis/redis-benchmarks-specification#527 · Opened Aug 22, 2026
[SPEC]: variadic generic commands (DEL/UNLINK/EXISTS/TOUCH) benchmarked only at width 1; MSETNX has no coverage Open

Difficulty 4/5 3-5 days Newbie friendliness 64/100

redis/redis-benchmarks-specification#526 · 1 comment · Opened Aug 22, 2026
[SPEC]: numkeys family — 8 commands with zero coverage; the 2 existing suites are pinned at the minimum width Open

Difficulty 4/5 3-5 days Newbie friendliness 70/100

redis/redis-benchmarks-specification#525 · Opened Aug 22, 2026
Add benchmark coverage for module command filters (RM_CommandFilter path) Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

redis/redis-benchmarks-specification#520 · Opened Aug 20, 2026
zremrangebyscore-pipeline-10 drains its 1M-element zset within ~100 calls, then measures a no-op fast path Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

redis/redis-benchmarks-specification#515 · Opened Aug 18, 2026
Extract shared CLEAN SHUTDOWN / MEMORY CEILING boilerplate from blocking-command spec descriptions Open

Difficulty 3/5 1-2 days Newbie friendliness 58/100

redis/redis-benchmarks-specification#511 · Opened Aug 17, 2026
Add benchmark coverage for SPOP with count (random-extraction strategies + batched SREM propagation) Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

redis/redis-benchmarks-specification#503 · Opened Aug 17, 2026
Add TLS-enabled benchmark coverage — the TLS connection layer is entirely unmeasured Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

redis/redis-benchmarks-specification#498 · Opened Aug 17, 2026
Add benchmark coverage for eviction under maxmemory pressure Open

redis/redis-benchmarks-specification#497 · 1 comment · 1 assignee · Opened Aug 16, 2026
Add benchmark coverage for glob pattern matching (KEYS, SCAN MATCH, PSUBSCRIBE) Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

redis/redis-benchmarks-specification#496 · Opened Aug 15, 2026
100Kstreams-xreadgroup suite: 100% of XREADGROUP ops return -NOGROUP on a sharded deployment, and the suite still passes Open

redis/redis-benchmarks-specification#494 · 2 comments · 1 assignee · Opened Aug 13, 2026
clientconfigs: no start ordering / readiness barrier between concurrent clients Open

Difficulty 4/5 3-5 days Newbie friendliness 46/100

redis/redis-benchmarks-specification#493 · Opened Aug 13, 2026
session-caching-hash-100k-sessions: clientconfig HSET reuses __key__ 8 times, scrambling userId/organizationId/etc across unrelated sessions Open

Difficulty 1/5 Under an hour Newbie friendliness 90/100

redis/redis-benchmarks-specification#490 · Opened Aug 13, 2026
1key-set-10-elements-smembers-pipeline-10 runs near load-generator saturation (memtier at 81% of its threads) Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

redis/redis-benchmarks-specification#487 · Opened Aug 12, 2026
feature-store serving suite is load-generator bound (memtier at 91% of its threads); ingest sibling uses -t 10, serving uses -t 4 Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

redis/redis-benchmarks-specification#486 · Opened Aug 12, 2026
Add benchmark coverage for template-encoded hash field add/remove (multi-field HSET / HDEL) Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

redis/redis-benchmarks-specification#481 · Opened Aug 10, 2026
Add benchmark coverage for MSETEX (multi-key string write with TTL) Open

Difficulty 2/5 1-2 days Newbie friendliness 78/100

redis/redis-benchmarks-specification#480 · Opened Aug 8, 2026
stream cgroups specs do not reach streamEntryIsReferenced()/streamCleanupEntryCGroupRefs(): XTRIM/XDEL default to KEEPREF Open

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

redis/redis-benchmarks-specification#479 · Opened Aug 8, 2026
Remaining uncovered bitmap write paths after #476 Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

redis/redis-benchmarks-specification#478 · Opened Aug 8, 2026
Two bitmap read specs measure the missing-key fast path (~90% misses) Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

redis/redis-benchmarks-specification#477 · Opened Aug 8, 2026
Add benchmark coverage for vector sets (VADD/VSIM paths) Open

Difficulty 4/5 3-5 days Newbie friendliness 50/100

redis/redis-benchmarks-specification#468 · Opened Aug 2, 2026
Add benchmark coverage for stream delete strategies (XDELEX / XACKDEL, DELREF and ACKED paths) Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

redis/redis-benchmarks-specification#467 · Opened Aug 2, 2026
redis-benchmarks-spec console script cannot start: the __spec__ subpackage shadows the ModuleSpec CPython's importer reads Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

redis/redis-benchmarks-specification#466 · 1 comment · Opened Aug 2, 2026
--use-git-timestamp is discarded before it reaches the stream (hardcoded True), so commit-dating cannot be turned off Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

redis/redis-benchmarks-specification#464 · Opened Aug 2, 2026
compare: --*-target-version / --*-target-branch build a filter that can never match, so they silently select nothing Open

Difficulty 3/5 1-2 days Newbie friendliness 56/100

redis/redis-benchmarks-specification#462 · Opened Aug 2, 2026
'ref' is written to the commits stream but never read (adjacent 'ref_label' is) — dead field Open

Difficulty 2/5 1-3 hours Newbie friendliness 76/100

redis/redis-benchmarks-specification#458 · Opened Jul 31, 2026
admin cancel: --tests-regexp is used as a literal LREM value, not a regex — patterns silently remove nothing Open

Difficulty 3/5 1-2 days Newbie friendliness 58/100

redis/redis-benchmarks-specification#457 · Opened Jul 31, 2026
use_git_timestamp is written as str(bool) and parsed with bool(str), so 'False' is always True — datapoints are unconditionally commit-dated Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

redis/redis-benchmarks-specification#456 · Opened Jul 31, 2026
--dockerhub path passes git_version into the git_hash positional; git_hash gets a version string and git_version is None Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

redis/redis-benchmarks-specification#455 · Opened Jul 31, 2026
API writes gh_org/gh_repo to the commits stream but consumers read github_org/github_repo — silently falls back to the 'redis' default Open

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

redis/redis-benchmarks-specification#453 · Opened Jul 31, 2026
Two stream fields are read but never written: 'executable' (dead branch) and 'docker_air_gap' (per-test override unreachable) Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

redis/redis-benchmarks-specification#450 · Opened Jul 31, 2026
--tests-regexp matches the filename stem in the coordinator but the 'name:' field in the CLI, with different anchoring — silently under-selects Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

redis/redis-benchmarks-specification#449 · 1 comment · Opened Jul 31, 2026
Add benchmark coverage for sharded Pub/Sub (SSUBSCRIBE / SPUBLISH path) Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

redis/redis-benchmarks-specification#447 · Opened Jul 30, 2026
Add benchmark coverage for RESTORE command Open

Difficulty 3/5 1-2 days Newbie friendliness 66/100

redis/redis-benchmarks-specification#433 · 1 comment · Opened Jul 1, 2026
Add benchmark coverage for replication client compression (repl-compression / zstd) Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

redis/redis-benchmarks-specification#429 · Opened Jun 24, 2026
Add benchmark coverage for LSET (list element replacement path) Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

redis/redis-benchmarks-specification#422 · Opened Jun 14, 2026

redis/redis-benchmarks-specification

Get new issues in your inbox