ruvnet / ruvnet/RuVector

[PIR][WP13] Implement closed-form KV-cache cross-model migration in ruvllm (arXiv:2608.03893)

Open
#844 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

fast-follow pir
Dominant language
Rust
Stars
4.5k
Forks
603
Avg merge
23h 32m
Merged PRs (30d)
59

Description

Epic: #837 · Plan: docs/research/perpetual-intelligence-runtime/03-program-plan.md

Goal

Implement the closed-form linear KV-cache mapper for same-family model migration in crates/ruvllm (kv_cache.rs, paged_attention.rs, serving/kv_cache_manager.rs), with the nonlinear MLP fallback for degrading pairs, plus a routing gate that predicts transfer quality before migrating — so the system never migrates blind.

Why this ships first

Grade-A evidence, cheapest to build, and it depends only on ruvllm. No dependency on WP0–WP12 — it runs as a parallel track and can start immediately, landing while the main branch is still in Phase 0/1.

Evidence

arXiv:2608.03893 — "Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse," submitted 2026-08-04. The strongest-evidence item in the whole program review, exact match on every figure:

  • The mapper runs 2.7–25x faster than re-prefill.
  • Six pairs tested across three model families.
  • The linear mapper retains 73–98% of the receiver's standalone-prefill accuracy on four pairs, while two degrade sharply.
  • A nonlinear MLP variant recovers up to +37pp HellaSwag accuracy on the failing pairs.

The 2-of-6 failure rate is the point: it is exactly why the quality-prediction gate is a requirement rather than a nice-to-have.

Acceptance criteria

  • Reproduces the paper's reported 2.7–25x speedup over re-prefill on an internal same-family model pair. Report our own measured numbers, never the paper's restated as ours.
  • The routing gate correctly refuses or downgrades for a known-degrading pair.
  • Closed-form linear mapper implemented for same-family migration.
  • Nonlinear MLP fallback implemented for pairs the linear mapper degrades on.
  • No code path migrates without consulting the gate. Enforced structurally, with a bypass-attempt test.
  • Per-pair accuracy retention measured and recorded; degrading pairs are identified automatically rather than hand-maintained in a list that will drift.
  • Degradation is observable at runtime — a migration whose realized quality falls below prediction is logged and counts against the gate's calibration.
  • Published as @ruvector/kv-migrate per the existing @ruvector/* release flow.

Scope boundary

Cross-provider routing (Anthropic/OpenAI/local) is not in scope — that role sits closer to claude-flow's ADR-026 3-tier routing. This is same-family KV-cache transfer within ruvllm's serving runtime.

Dependencies

None. Independent parallel track, can start immediately.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/research/perpetual-intelligence-runtime/03-program-plan.md and inspect crates/ruvllm/kv_cache.rs, paged_attention.rs, and serving/kv_cache_manager.rs, then compare the paper's mapper and failure cases with the existing serving flow. Done means the gate, linear mapper, fallback, structural bypass test, measurements, observability, and @ruvector/kv-migrate release criteria are demonstrated without adding cross-provider routing.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.