SOTA experiment: CacheBridge cross-model KV transfer for router handoffs

Finding

CacheBridge: Efficient Cross-Model KV Cache Transfer (arXiv:2609.00891, submitted 2026-09-01) reports cross-model KV transfer intended to avoid full receiver prefill during model-family handoffs. The originating team reports 99.83% mean target retention on Qwen3 in one direction family, up to 3x faster mapper application, 8x lower mapper storage, matching with one tenth the calibration data, and a reported mapper construction reduction from 92.63s to 8.63s in a Qwen3 14B to 32B setting.

Evidence status: originating-team measured, GPU/model-family specific, not independently reproduced by RuV.

RuV opportunity

This maps to ruvLLM, RuVector, Cognitum model routing, MidStream, and distributed inference. It could make mid-session model escalation materially cheaper if cache transfer preserves quality.

Experiment

Do not replace prefill globally. Add an isolated adapter benchmark with:

1. normal target re-prefill
2. prior simple cross-model mapping baseline
3. CacheBridge-style head-matched mapping

Pin model revisions, tokenizer, RoPE configuration, context lengths, CUDA, driver, PyTorch/serving runtime, GPU, calibration corpus, and random seeds.

Metrics

• target task accuracy / perplexity retention
• TTFT and p50/p95 handoff latency
• mapper construction time
• mapper storage
• calibration sample count
• GPU memory and transfer bandwidth
• failure by context length and transfer direction
• quality after multiple model switches

Falsification

Mandatory controls include just re-prefilling, prefix caching on the same model, and a simpler linear map. If handoff frequency is low or transferred quality falls materially, the added mapping layer should be rejected.

Promotion gate

At least 2x lower handoff prefill latency on a real RuV routed workload, target quality loss no greater than 1 absolute point, deterministic rollback to ordinary prefill, and no change to RVM authority or provenance semantics.

No files, tests, or entry points are named. Start by locating the existing model-routing and adapter benchmark entry points, then compare ordinary re-prefill, the prior simple mapping baseline, and CacheBridge-style head-matched mapping under the pinned conditions. Done means the required latency, quality, storage, calibration, memory, bandwidth, context-length, direction, and multi-switch metrics meet the stated promotion gate with rollback preserved.