port(KV): stage 6 of the KV-cache layout refactor is IN the pin target at queue position 230, and two records say it is not
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: none exists. No row in .agents/roadmap_v1.md or any *-matrix.md owns the KV-cache layout refactor series. This is the same absence #2695 records for stage 5 and #2693 for stage 4. A series row is warranted; this issue does not invent one. The nearest existing rows are KV-DSV4-MULTICACHE (.agents/engine-matrix.md:113) and KV-HYBRID-COORD (:100), neither of which owns AttentionSpec.
Found by wave PORTQ-6 (#2718) re-deriving PORT-NOW entry [230], upstream 8bdc70ec7b vllm#51718, of 5559679229..e126687a9a. Nothing was executed.
First: stage 6 IS inside the pin target, and two records say otherwise
git merge-base --is-ancestor 8bdc70ec7b e126687a9a succeeds; --is-ancestor 8bdc70ec7b 5559679229 fails. It is in range, and it is PORT-NOW queue position 230, alongside stage 4 at 145 and stage 5 at 167. All three stages are in range and all three are PORT-NOW entries.
.agents/sync/2026-09-02-db92053.md:131 says stage 6 is out of range. That file needs nothing — it is already annotated at :137-149 (PORTQ-4, #2680), and the annotation is correct: the table is true at that report's target db92053e97 (2026-08-19), where 8bdc70ec7b (2026-08-21) genuinely postdates it.
.agents/sync/2026-09-03-portq5.md §5.3 is the one that is false, and this issue's companion pull request annotates it. It states the out-of-range claim flatly, with no target qualifier, at its own target e126687a9a. Its citation db92053.md:127-144 spans lines 137-144 of the very annotation that corrects it — the correction landed in 447647ca4 and PORTQ-5's report in 69f08a65a, two minutes and nineteen seconds later.
This issue's title and scheduling note supersede #2695's. "Must not land alone because stage 6 is out of range" should read "must land as a 4 → 5 → 6 sequence that reaches stage 6, all three now in range". The sequencing constraint is untouched and real; what changed is that it is now satisfiable inside this pin advance.
What upstream does
vllm/v1/kv_cache_layout.py is new: a KVCacheLayout enum whose six members are stride permutations over logical [L, B, H, N, C], with stride_order / layer_view_order / is_layer_compact / is_block_contiguous / is_block_compact / is_block_outermost.
kv_cache_interface.py (+242) deletes storage_block_size in favour of num_states plus get_num_kernel_states(kernel_block_size); adds num_heads, tokens_per_state, state_content_size_bytes to KVCacheSpec; drops indexes_kv_by_block_stride from AttentionSpec; drops compress_ratio from MLAAttentionSpec and SlidingWindowMLASpec in favour of tokens_per_state; adds the module functions group_kernel_blocks, compute_layer_kv_cache_shape_bytes, compute_layout_strides and create_kv_cache_views (the 5D as_strided view plus per-layer 4D views, with the padded-page and kernel-block-split guards).
attention/backend.py (−108) deletes the get_kv_cache_shape abstract method, get_kv_cache_block_dim, get_kv_cache_stride_order and indexes_kv_by_block_stride from the backend ABC, and turns get_required_kv_cache_layout into supported_kv_cache_layouts(). Roughly twenty-five backends then delete their own overrides. config/cache.py (+40) gains kv_cache_layout and the NHD -> LBNHC / HND -> LBHNC aliases; envs.py widens VLLM_KV_CACHE_LAYOUT. worker/utils.py (+172), gpu/attn_utils.py (−392) and gpu_model_runner.py (−280) collapse per-backend view construction onto create_kv_cache_views.
What is here: the pre-commit shape, and it is reached from production
include/vllm/v1/kv_cache_interface.h:130—virtual int storage_block_size() const { return block_size; }, overridden at:327and:427asblock_size / compress_ratio. 32 mentions, consumed by the page-size math atsrc/vllm/v1/kv_cache_interface.cpp:135,151,164,179andsrc/vllm/v1/core/hybrid_kv_budget.cpp:57.compress_ratioas the spec field (as opposed to the ~130 model-internal DSv4/QSA hits, which do not move): 11 lines inkv_cache_interface.h, 4 inkv_cache_interface.cpp, 2 inkv_cache_coordinator.cpp(the group-equality test), plus the registries.indexes_kv_by_block_stride: 31 sites — the field atkv_cache_interface.h:157, six constructor chains, five group-equality clauses inkv_cache_coordinator.cpp:29,52,69,84,95, two call sites inkv_cache_utils.cpp:66,76, three registry defaults, andchunked_local_attention.{h:124,cpp:139,145}.- The deleted ABC method is still here and is reached:
include/vllm/v1/attention/backend.h:295declaresget_kv_cache_shapepure virtual, with overrides atbackend.h:433,555,587,backend.cpp:238,248,263,cpu_attn.cpp:19,gdn_attn.cpp:402,chunked_local_attention.cpp:69.src/vllm/v1/attention/registry.cpp:150-178CheckKvCacheShapecalls it and throws on mismatch, fromsrc/vllm/v1/worker/gpu/runner.cpp:1629. This is a production refusal path, not a class. - None of the new side exists. See the zeros below.
One design step is not transcription. vt::Tensor is rank-4 max (include/vt/tensor.h:12, constexpr int kMaxRank = 4) with an elements-not-bytes stride array, so upstream's 5D as_strided cannot be expressed directly; create_kv_cache_views has to compute the five byte strides on the host and emit per-layer 4D views.
The physical layout is hard-wired NHD and asserted in the kernels. src/vt/ops.cpp:3975, :4033 and :4898 each throw on "stride[2] == head_size — the NHD unbind-slice layout"; src/vt/cuda/cuda_paged_attn.cu:1554, src/vt/cpu/cpu_cache.cpp:41 and src/vt/metal/metal_msl.h:833 assume the same. LBNHC is the only reachable member, so the port must refuse the other five by name rather than shipping five dead enum values.
Size, and the series total
~900-1400 product lines across ~25-35 files, plus ~300-600 test lines. Roughly: spec migration ~200; layout descriptor ~70; geometry functions ~200-250 (longer than upstream's ~130 Python lines for the rank-4 reason above); backend ABC plus the refusal path ~150-200; runner rewiring ~150-300; config, env and resolution ~100-140. The runner figure is the softest number here.
Against the series: stage 4 (#2693) ~80-140 lines, stage 5 (#2695) ~250-400 lines, stage 6 ~900-1400. Series total ~1250-1950 product lines plus ~400-800 test, across ~35-45 files, and it rewrites a production refusal path and the runner's KV allocation. That is not one branch. .agents/sync/2026-09-01-cdefd9d.md §12 sizes stage 6 at ~150-250 lines by scoping it to one of its three parts; this issue sizes the whole commit, and the difference is the ABC change and the runner rewiring, not disagreement about the KVCacheLayout part.
Consequence of not porting
Nothing user-visible breaks today. The tree computes correct NHD pages and the DSv4/QSA compression math is right. The cost is structural: KV geometry stays spread across seven backend overrides and the runner instead of one seam; compress_ratio and storage_block_size keep diverging from every upstream anchor cited beside them; and every later upstream KV commit lands on a shape upstream no longer has.
It reverts [229], and a later commit restores it
8bdc70ec7b reverts entry [229] 9ff7041b55's flash_attn.py builder hunk back to model-config geometry, and entry [241] f620499ee3 (PORTQ-7's tranche, titled "Reapply") restores it. 9ff7041b55 is an ancestor of 8bdc70ec7b, so the revert is real and not a rebase artifact. At the target, flash_attn.py:436-441 carries [229]'s shape. Anyone taking these in SHA order must take all three or none.
Zeros, with their controls
Scope src/ and include/, probe form grep -rn <needle> src include | wc -l, one needle per call.
KVCacheLayout 0 · VLLM_KV_CACHE_LAYOUT 0 · create_kv_cache_views 0 · group_kernel_blocks 0 · get_num_kernel_states 0 · num_head_slots 0 · state_content_bytes 0 · state_content_size_bytes 0 · get_page_sizes 0 · layer_stride 0 · resolve/set/get_kv_cache_layout 0.
Three are not zeros and would have been read as one: kv_cache_layout returns 1 (a comment, dots3_note_attn.h:366), stride_order 1 (backend.h:35), unpadded_page_size_bytes 1 (kv_cache_interface.cpp:173). Two are false friends: num_states returns 5, all a local size_t in runner.cpp:957-962; tokens_per_state returns 10, all the QSA model-local field, and include/vllm/model_executor/models/qwen4_exp_qsa.h:169-170 says so explicitly — neither is the spec property.
Positive controls through the identical form: page_size_bytes 133 · AttentionSpec 241 · MLAAttentionSpec 91 · block_size 1739 · storage_block_size 32 · compress_ratio 300 · indexes_kv_by_block_stride 31 · get_kv_cache_shape 50.
Pre-pin
git show 5559679229:vllm/v1/kv_cache_interface.py has storage_block_size at :119 and :394, indexes_kv_by_block_stride at :182, compress_ratio at :386, head_size_v at :237 — exactly the shape this tree mirrors. The tree is pin-current on the surface stage 6 rewrites, so this port is owed by this pin advance. vllm/v1/kv_cache_layout.py is absent at the pin; this commit creates it.
One caveat: stage 5's num_head_slots / state_content_bytes are missing here (#2695), so stage 6 cannot be applied as a diff against the tree. It has to be applied on top of stage 5's result.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with upstream commit 8bdc70ec7b and the existing KV interfaces in include/vllm/v1/kv_cache_interface.h, attention/backend.h, and worker/gpu/runner.cpp; confirm the stage-5 prerequisites from #2695 first. Trace the listed production refusal path and current NHD assumptions before mapping the rank-4 views to the upstream layout. Done means stages 4→5→6 are ported in order, the reachable layout path is updated, and the five unsupported layouts are refused by name.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 30/100