mudler / mudler/vllm.cpp

MODEL-DSV4: DeepseekV4PagedSwaPages indexes attn_kv with MultiKvCacheIndex::Find's FLAT index, which is a paged slot only because that model publishes no recurrent group

Open
#2,456 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: MODEL-DSV4-DSA-COMPOSE

Found while repairing the same defect on MODEL-MM-GLM53-FLASH (#2445). Filed rather than fixed in flow, for the reasons under ## Owed below.

The two indices MultiKvCacheIndex answers, and which one attn_kv takes

DeepseekV4PagedSwaPages (src/vllm/model_executor/models/deepseek_v4.cpp:3415) reads

const int64_t idx = multi_kv.Find(name);
if (idx < 0 || idx >= static_cast<int64_t>(attn_kv.size())) { ... }
const PagedKvCache& c = attn_kv[static_cast<size_t>(idx)];

MultiKvCacheIndex::Find answers the FLAT index — the cache's place among every published cache, paged and recurrent together, in publication order. Its own header says so: "it is NOT a slot in either payload container, and on any topology carrying a recurrent group it is not equal to one either" (include/vllm/model_executor/models/model_registry.h:482-489). The slot in attn_kv comes from PayloadAt / Resolve, off the payload_slots vector 9e7621efc added.

This is not a live defect today, and the reason is the topology and nothing else

MakeDeepseekV4KVCache publishes no MambaSpec group. Every published cache on that model is paged, so paged_slot increments once per entry, the flat index and the paged slot are equal for every one of them, and Find and PayloadAt cannot disagree. The code is correct by accident of the model's shape, not by contract.

Why it is worth an issue anyway

The same line on glm5_next threw at the first step because 45 happens to be out of range for a 22-entry vector. That is an accident of ordering. On a topology whose recurrent group is published before the last attention group but whose paged count is large enough, the flat index is in range and names a different layer's cache — a plausible float buffer of the right rank and, where two attention groups share a page geometry, the right shape too. That is a fluent-wrong-tokens failure with no throw, which is the class MODEL-DSV4-DSA-COMPOSE already refuses elsewhere by name.

The fix

Mirror what glm5_next_kv.cpp now does: resolve through PayloadAt, refuse a kRecurrent payload under an attention name by name, and index attn_kv by the returned slot. Keep reading group_ids at the flat index, because that vector is parallel to the flat list. src/vllm/model_executor/models/qwen4_exp_registry.cpp:411-440 is the pattern; src/vllm/model_executor/models/glm5_next_kv.cpp is the second instance.

The test that would have caught it is the same shape too: deepseek_v4's fixture publishes its channel the way the runner did before 9e7621efc, so it cannot express a flat index that differs from a paged slot.

Owed

Not fixed in flow. It is a different row's file, it produces no wrong answer at any topology this tree can build today, and the change wants that row's own sibling-inertness gate rather than the GLM-5.3-Flash one. Listed under ## Owed in .agents/specs/glm5-next-flash.md as O37, which names MODEL-DSV4-* as the owner.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/vllm/model_executor/models/deepseek_v4.cpp:3415, then read MultiKvCacheIndex in include/vllm/model_executor/models/model_registry.h and compare the resolution pattern in src/vllm/model_executor/models/qwen4_exp_registry.cpp:411-440 and src/vllm/model_executor/models/glm5_next_kv.cpp. Update the DeepSeek V4 lookup and its fixture so a mixed paged/recurrent topology is represented; done means attention names resolve to the correct paged slot and the relevant tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.