SOTA: dynamic KV capacity and temporal ranking benchmark

Finding

Two September 3 results challenge fixed per request KV budgets and repeated full scorer refresh as default assumptions for long reasoning serving.

GrowPage treats KV capacity as an on demand runtime resource that can acquire additional pages or compress at page boundaries while retaining PagedAttention compatibility. What Matters for Aggressive Decoding Time KV Eviction reports that temporal aggregation and ranking preservation can dominate scorer sophistication under aggressive compression, with InertiaKV Lazy reporting 1.34 to 1.46 times decode throughput relative to full refresh and a score free variant reporting average quality change of plus 0.03 while eliminating subsequent scoring.

Evidence class: originating team measured, RuV reproduction pending.

Opportunity Score: 4.385.

RuV mapping

RuVector and ruvLLM own cache representation and policy. Cognitum can use the resulting telemetry for placement and routing. MidStream and LatentMesh benefit when long sessions or edge cloud handoffs are constrained by KV memory rather than compute.

Experiment

Build one common KV benchmark rather than another production policy. Compare:

1. Current ruvLLM cache policy.
2. Current policy plus dynamic page acquisition under a fixed global memory pool.
3. Current policy plus lazy temporal ranking refresh.
4. Combined dynamic capacity plus lazy ranking.
5. Existing TwinKV and any available HeadWiseKV or CacheBridge candidate under the same harness.

Benchmark contract

Pin model, tokenizer, serving runtime, GPU, driver, CUDA, cache block size, batch concurrency, prefix reuse, workload seeds, prompt lengths, output lengths, memory cap, and admission policy.

Report task quality, tokens per second, time to first token, inter token latency, p50 and p99 completion latency, peak KV memory, cache hit rate, evictions, recomputations, page acquisitions, ranking refresh count, scheduler overhead, energy where measurable, failures, OOMs, and cost.

Test burst concurrency, adversarial long outputs, rapid prefix churn, tiny and huge sessions, memory pressure, malformed telemetry, and starvation.

Promotion gates

A candidate advances only with at least 20 percent higher throughput or at least 25 percent lower peak KV memory at matched quality, p99 latency regression below 5 percent, zero starvation, and no more than 1 absolute point quality loss on protected long context tasks.

Governance

No cache format migration or default switch until the benchmark is independently reproduced. Current cache behavior remains rollback. Routing score or cache pressure never grants execution authority.