port(KV): the KV-cache layout refactor stages 4-6 are one unit, and all three are inside the current pin target
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: none exists. No row in .agents/roadmap_v1.md or any *-matrix.md owns the KV-cache layout refactor series. The nearest existing rows are KV-DSV4-MULTICACHE (.agents/engine-matrix.md:113), which owns the MLA page-size constants this touches but not AttentionSpec, and KV-HYBRID-COORD (:100). A series row is warranted; this issue does not invent one.
Found by wave PORTQ-5 (#2679) re-deriving PORT-NOW entry [167], upstream 57bd0ed441 vllm#51704, stage 5 of the series. Nothing was executed.
What is portable today
Upstream hoists head_size_v from the subclasses onto AttentionSpec; adds backend-published num_head_slots / state_content_bytes and the derived num_heads / state_content_size_bytes; collapses five per-subclass real_page_size_bytes overrides into one unpadded_page_size_bytes = num_heads * storage_block_size * state_content_size_bytes; and adds the AttentionBackend.customize_spec(spec) -> spec hook.
Against this tree:
head_size_vis still per-subclass —include/vllm/v1/kv_cache_interface.h:139-157(base members stop atindexes_kv_by_block_stride),:239(Full),:352(SlidingWindow).- Five separate
real_page_size_bytes()overrides carry the page-size math:src/vllm/v1/kv_cache_interface.cpp:83-91,:107-117,:122-153(MLA, with the 584/656 fp8_ds_mla constants),:158-181,:183-193. Nounpadded_page_size_bytesfunction exists; the only occurrence of that string is a prose comment at:173. AttentionBackendalready carries classmethod-shaped virtual hooks (include/vllm/v1/attention/backend.h:286-300,:356-365), socustomize_spechas a home.
What is NOT portable
The TQFullAttentionSpec deletion and the four turboquant KVQuantMode members are surface-absent. KVQuantMode here stops at kNvfp4 = 5 (include/vllm/v1/kv_cache_interface.h:108-115) — which matches the pin exactly — and include/vllm/v1/kv_cache_dtype.h:86-89 refuses turboquant_* outright. Flipping them would land on nothing.
The scheduling constraint, which is the point of this issue
.agents/sync/2026-09-02-db92053.md:127-144 records it: stages 4 (61874f9842) and 5 (this one) are inside 5559679229..e126687a9a, and stage 6 (8bdc70ec7b) is not. Porting 4 and 5 without 6 lands a half-migrated layout surface that no upstream revision ever had.
So this entry must not land alone. It is a three-stage unit preceded by two preparatory steps the sync records already name:
- Bring
unify_hybrid_kv_cache_specs(src/vllm/v1/core/kv_cache_utils.cpp:26-93) up to pin state (db92053:1876-1887). - Convert the positional spec constructors to designated initializers before any stage —
head_size_v,sliding_windowandattention_chunk_sizeare allint, so the hoist silently re-binds positional call sites across ~8 model registries (cdefd9d:1867-1875).
Then either the pin target moves to 8bdc70ec7b or later, or the wave stops before stage 4.
Reachability
customize_spec needs a production call site as well as a virtual. Today the spec is built by model registries (e.g. src/vllm/model_executor/models/qwen3_5_common.cpp:72-90) which never consult a backend. Without that call site the hook lands dead, per AGENTS.md §"Nothing lands dead".
Size
~250-400 lines across include/vllm/v1/kv_cache_interface.h, src/vllm/v1/kv_cache_interface.cpp, include/vllm/v1/attention/backend.h, the ~8 registry call sites and tests/vllm/v1/test_kv_cache_interface*. Larger than the "4 core + 2 test files" .agents/sync/2026-09-01-cdefd9d.md:1846 estimates, because the constructor conversion is not in that count.
Open question, carried forward not resolved
cdefd9d:1901-1907 asks whether state_content_bytes is the right home for kFp8DsMlaV4TokenBytes / kFp8DsMlaV32TokenBytes. Those constants are at src/vllm/v1/kv_cache_interface.cpp:135,139,164 and are still selected by cache_dtype_str rather than by a backend hook, so the question is still open. It needs reading how vllm/models/deepseek_v4/attention.py publishes those bytes after stage 5.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .agents/sync/2026-09-02-db92053.md:127-144 and the preparatory changes in src/vllm/v1/core/kv_cache_utils.cpp and the model registry constructors. Read the listed kv_cache_interface, attention/backend, and test files, then inspect deepseek_v4/attention.py for state-content bytes. Done means stages 4-6 land together at the required pin, customize_spec has a production call site, and the KV-cache tests cover the migrated layout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- ai-infra-agents, backend, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100