[PIR][WP13] Implement closed-form KV-cache cross-model migration in ruvllm (arXiv:2608.03893)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 4.5k
- Forks
- 603
- Avg merge
- 23h 32m
- Merged PRs (30d)
- 59
Description
Epic: #837 · Plan: docs/research/perpetual-intelligence-runtime/03-program-plan.md
Goal
Implement the closed-form linear KV-cache mapper for same-family model migration in crates/ruvllm (kv_cache.rs, paged_attention.rs, serving/kv_cache_manager.rs), with the nonlinear MLP fallback for degrading pairs, plus a routing gate that predicts transfer quality before migrating — so the system never migrates blind.
Why this ships first
Grade-A evidence, cheapest to build, and it depends only on ruvllm. No dependency on WP0–WP12 — it runs as a parallel track and can start immediately, landing while the main branch is still in Phase 0/1.
Evidence
arXiv:2608.03893 — "Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse," submitted 2026-08-04. The strongest-evidence item in the whole program review, exact match on every figure:
- The mapper runs 2.7–25x faster than re-prefill.
- Six pairs tested across three model families.
- The linear mapper retains 73–98% of the receiver's standalone-prefill accuracy on four pairs, while two degrade sharply.
- A nonlinear MLP variant recovers up to +37pp HellaSwag accuracy on the failing pairs.
The 2-of-6 failure rate is the point: it is exactly why the quality-prediction gate is a requirement rather than a nice-to-have.
Acceptance criteria
- Reproduces the paper's reported 2.7–25x speedup over re-prefill on an internal same-family model pair. Report our own measured numbers, never the paper's restated as ours.
- The routing gate correctly refuses or downgrades for a known-degrading pair.
- Closed-form linear mapper implemented for same-family migration.
- Nonlinear MLP fallback implemented for pairs the linear mapper degrades on.
- No code path migrates without consulting the gate. Enforced structurally, with a bypass-attempt test.
- Per-pair accuracy retention measured and recorded; degrading pairs are identified automatically rather than hand-maintained in a list that will drift.
- Degradation is observable at runtime — a migration whose realized quality falls below prediction is logged and counts against the gate's calibration.
- Published as
@ruvector/kv-migrateper the existing@ruvector/*release flow.
Scope boundary
Cross-provider routing (Anthropic/OpenAI/local) is not in scope — that role sits closer to claude-flow's ADR-026 3-tier routing. This is same-family KV-cache transfer within ruvllm's serving runtime.
Dependencies
None. Independent parallel track, can start immediately.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with docs/research/perpetual-intelligence-runtime/03-program-plan.md and inspect crates/ruvllm/kv_cache.rs, paged_attention.rs, and serving/kv_cache_manager.rs, then compare the paper's mapper and failure cases with the existing serving flow. Done means the gate, linear mapper, fallback, structural bypass test, measurements, observability, and @ruvector/kv-migrate release criteria are demonstrated without adding cross-provider routing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100