mudler / mudler/vllm.cpp

port(KV-PREFIX-MATCH-UNIT): the sync target's KV rework is inert here except one alignment check, which is blocked on unported stage 6

Open
#2,736 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: KV-PREFIX-MATCH-UNIT (.agents/engine-matrix.md:60), which names resolve_kv_cache_block_sizes explicitly and records the resolver as LANDED with W2/W3/W4 deferred. Resolved by line number before filing.

Found by wave PORTQ-7 (#2717) re-deriving PORT-NOW entry [290], upstream e126687a9a vllm#53896 — the sync target itself, and the last entry of the 290-entry queue. Nothing was executed.

Why this issue is small when the entry is enormous

Entry [290] has two halves and they end up in different places.

Half (a), the qwen4_exp architecture, is a REAL_GAP that is already owned and already reconciled. Row MODEL-MM-qwen4-exp-qwen4-exp-for-conditional-generation (.agents/model-matrix.md:514, state ACTIVE), spec .agents/specs/qwen4-exp-flash-next.md, reconciled against this exact SHA on 2026-09-01 by Q4RECONCILE (#2489) with a 24-row component table, including two divergences the tree is wrong about (#2477, #2406) and the unimplemented MTP head. No new issue is filed for half (a) — filing one would be intake against work that already has an owner, a spec and three issues.

Half (b), the KV-grouping rework, does not do here what the queue line says it does. That is the finding, and it is what this issue records.

The queue line's premise for half (b) does not hold

.agents/sync/2026-09-01-cdefd9d.md describes half (b) as changing "KV grouping for models we already run", and :1850 sizes it "REAL_GAP for the generic parts | ~120-200 lines". Read against the tree, it splits five ways and only one small piece is portable:

  • iter_layer_specs in the mamba-mode check — a no-op. Every KVCacheGroupSpec here holds exactly one KVCacheSpec (src/vllm/v1/core/kv_cache_utils.cpp:657-661, :678-682), because UniformTypeKVCacheSpecs is deferred (include/vllm/v1/kv_cache_interface.h:48). Same for the identical change to has_mamba_layers (src/vllm/v1/kv_cache_interface.cpp:232-238).
  • The prefix-cacheable-only GCD — inert. Upstream's prefix_cacheable returns True on the base and False on exactly one subclass, the brand-new CircularBufferSpec. Every spec this tree registers (src/vllm/v1/kv_cache_spec_registry.cpp:71-92) is prefix-cacheable, so hashing_sizes == group_block_sizes and the GCD is unchanged. This is the direct answer: KV grouping for models this tree already runs does not move.
  • _get_packed_kv_cache_groups replacing group_and_unify_kv_cache_specs — surface-absent, by a wide margin. There is no generic grouping planner: each model registry builds its own groups (~30 kv.kv_cache_groups.emplace_back(...) sites). The function's first gate is layout.is_block_outermost, and KVCacheLayout -> 0, is_block_outermost -> 0, get_resolved_kv_cache_layout -> 0; this tree hard-wires one NHD view (positive control NHD -> 69, identical probe form and scope src/+include/).
  • CircularBufferSpec / CircularBufferManager / the _record_new_block_ids exclusion — surface-absent. All three -> 0; this tree records unconditionally (single_type_kv_cache_manager.cpp:155,175), and the exclusion is meaningless without the spec.
  • tp_replicated, MambaStateCopyFuncsByType, num_states flattening — surface-absent. All -> 0; get_mamba_groups -> 2, both prose citations of upstream's uniformity assert.

The portable piece, and what blocks it

The per-state prefix alignment checkprefix_alignments derived from tokens_per_state > 1, and the cache_hit_alignment it feeds in resolve_kv_cache_block_sizes (src/vllm/v1/core/kv_cache_utils.cpp:640-707, currently the pin shape).

It is blocked. tokens_per_state -> 10 hits, every one the model-local field in src/vllm/model_executor/models/qwen4_exp_qsa.{h:191,cpp:81}. No KVCacheSpec in this tree carries it. Upstream puts it on KVCacheSpec in stage 6, 8bdc70ec7b vllm#51718, which is in-range at PORT-NOW queue position 230 and unported. Porting this piece without that stage would mean inventing the field here first.

Size: ~30-50 product lines plus one test file — not the ~120-200 the prior reading estimated for the whole half. That estimate is corrected here.

Sequencing, which is the operative constraint

Half (b) is the fourth stage of a series whose stages 4, 5 and 6 are all in-range, all unported, and all REAL_GAP:

stage upstream queue position issue
4 61874f9842 vllm#51612 145 #2693
5 57bd0ed441 vllm#51704 167 #2695
6 8bdc70ec7b vllm#51718 230
this e126687a9a vllm#53896 290 this issue

Taking [290]'s KV half first would land the tail of a series whose head is missing. All four stages are inside the current target, which #2695's title and body still deny; that issue is corrected by comment.

What could not be determined

The size of the blocked piece is medium confidence: 8bdc70ec7b's kv_cache_interface.py hunk was not read in full, so how much of tokens_per_state / num_states / state_content_size_bytes must come with it is open. Reading that hunk settles it.

The 24-row qwen4_exp reconciliation table was relied on as a committed record dated two days ago and not re-verified against the current tree. Its own "what this pass could NOT determine" section should be read before anyone treats it as complete.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the stage 6 upstream kv_cache_interface.py hunk first, then inspect resolve_kv_cache_block_sizes in src/vllm/v1/core/kv_cache_utils.cpp and the tokens_per_state fields in src/vllm/model_executor/models/qwen4_exp_qsa.h and .cpp. The work is done when the required dependency and per-state prefix-alignment check are ported with the relevant test file passing; the exact dependency scope remains to be determined.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.