mudler / mudler/vllm.cpp

port(KV-CHUNKED-LOCAL-SPEC): hybrid spec promotion is behind the pin on four counts, and it silently downgrades a SlidingWindowMLASpec

Open
#2,693 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: KV-CHUNKED-LOCAL-SPEC

Found by wave PORTQ-4 (#2680)
re-deriving PORT-NOW entry 145 of 5559679229..e126687a9a, upstream
61874f9842 vllm#51612
"[4/N][KV-Cache Layout Refactor] Promote local KV cache specs via a class-changing
replace helper".

The commit cannot be applied as a diff, because the function is behind the pin

This tree's promoter is unify_hybrid_kv_cache_specs
(src/vllm/v1/core/kv_cache_utils.cpp:26-93, declared
include/vllm/v1/core/kv_cache_utils.h:82). Read against
git show 5559679229:vllm/v1/core/kv_cache_utils.py:1425-1500, it is behind the
pin on four counts:

  1. No full_attention_block_size recompute — sliding->block_size and
    chunked->block_size are reused verbatim (kv_cache_utils.cpp:63,72).
  2. No promoted_page_size_padded recompute — page_size_padded is copied through
    unchanged (:66,74).
  3. No SlidingWindowMLASpec branch.
  4. Upstream's first guard is is_kv_cache_spec_uniform(...) or UniformTypeKVCacheSpecs.is_uniform_type(...); this tree has only the second half
    (are_uniform_kv_cache_specs, src/vllm/v1/kv_cache_spec_registry.cpp:149).
    is_kv_cache_spec_uniform is 0 hits over src/ and include/, against a
    positive control of KVCacheSpecKind at 77 hits.

So 61874f9842 is a refactor on top of a state this tree has not reached. It needs
a preparatory commit bringing the function to pin state first.

Point 3 is a live latent defect, and it still has no issue of its own

SlidingWindowMLASpec inherits SlidingWindowSpec
(include/vllm/v1/kv_cache_interface.h:397), so the
dynamic_cast<const SlidingWindowSpec*> at kv_cache_utils.cpp:60 matches it
and promotes it to a plain FullAttentionSpec, dropping cache_dtype_str,
alignment, compress_ratio and model_version.

It is unreachable today only because the function has no production caller
grep returns the definition, two declarations, one comment in
include/vllm/v1/core/hybrid_kv_budget.h, and ten sites in
tests/vllm/v1/test_kv_cache_utils.cpp. That makes it a trap for whoever wires the
caller. .agents/sync/2026-09-01-cdefd9d.md:1874-1891 named this; it was
re-verified at a700e8da6 by this wave and by its operator, and nothing has filed
it. This issue is where it now lives.

Nothing lands dead

Because the function has no production caller, a change that edits only it lands
nothing reachable. Whoever takes this must either wire the call site or name the
unreached slice, its owning row and its issue in both the commit body and the
pull request body, and list it under the row spec's ## Owed.

Sequencing — this is stage 4 of three, and the constraint now spans three tranches

61874f9842 is stage 4; stage 5 is 57bd0ed441
vllm#51704 and stage 6 is
8bdc70ec7b vllm#51718. All
three are inside 5559679229..e126687a9a and all three are PORT-NOW entries, at
queue positions 145, 167 and 230. Porting 4 and 5 without 6 lands a
half-migrated layout surface no upstream revision ever had.

.agents/sync/2026-09-02-db92053.md:129-144 states that hazard, but keyed to the
superseded db92053e97 target (2026-08-19), where stage 6 (2026-08-21) was out
of range; its conclusion "either it stops before stage 4, or the pin's target has to
be 8bdc70ec7b or later" is discharged by the target already having advanced. The
sequencing constraint itself stands. That record is annotated in place by this wave.

.agents/sync/2026-09-01-cdefd9d.md §12 hazard 1 adds one more: stage 5's
head_size_v hoist over this tree's positional, all-int, all-defaulted spec
constructors is a compiles-clean wrong-page-size failure across roughly eight model
registries. The named-parameter conversion should precede stages 4-6.

Size

~80-140 lines across src/vllm/v1/core/kv_cache_utils.cpp,
include/vllm/v1/kv_cache_interface.h and tests/vllm/v1/test_kv_cache_utils.cpp,
plus the un-sized preparatory commit. replace_as has no direct C++ form (no
dataclass reflection); it lands as a spec-to-spec converting constructor or as
designated-initializer aggregates.

Co-owning row for the sliding half: KV-SLIDING-WINDOW-SPEC
(.agents/engine-matrix.md:103, GATING, same local anchor).

Verification state

Nothing was executed — no build, no test run, no GPU, no lease. Every
path:line was printed from the tree at a700e8da6; the promoter body, the
SlidingWindowMLASpec inheritance and the caller census were re-read by the wave
operator. .agents/sync/2026-09-01-cdefd9d.md:1845 sized this ~80-140 lines, 2 src + 1 test and is corroborated.

Open question carried from the record

Whether stage 5's state_content_bytes is the right home for
kFp8DsMlaV4TokenBytes / kFp8DsMlaV32TokenBytes, given there is no
customize_spec call site here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/vllm/v1/core/kv_cache_utils.cpp:26-93, its declaration in include/vllm/v1/core/kv_cache_utils.h:82, and the related spec definitions in include/vllm/v1/kv_cache_interface.h:397. Read tests/vllm/v1/test_kv_cache_utils.cpp and the stage 5/6 sequencing notes before changing the promoter. Done requires the preparatory state, the four listed promotion gaps addressed, tests updated, and either a reachable caller or explicit ownership and Owed tracking for the unreached slice.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.