ModelRegistry::Forward's multi-cache refusal names ONE owner where THREE architectures now arrive, and never names the arriving architecture
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
ModelRegistry::Forward's multi_kv refusal (src/vllm/model_executor/models/model_registry.cpp:462-478) ends with:
(row KV-DSV4-MULTICACHE W5 owns the consuming forward; #1925, #2068)
That clause was true when W3 landed it, because DeepSeek-V4 was the only architecture that could publish a multi-cache topology. Three architectures reach the guard now, and the clause is false for two of them.
| architecture | groups | who owns its consuming forward |
|---|---|---|
DeepseekV4ForCausalLM |
7, all MLAAttentionSpec / SlidingWindowMLASpec |
KV-DSV4-MULTICACHE W5 -- the clause is true here |
Qwen4ExpForConditionalGeneration |
3 (FullAttentionSpec, MambaSpec, MLAAttentionSpec) |
MODEL-MM-QWEN4-EXP. Qwen4ExpTextModel::Forward does not exist |
Glm5NextForConditionalGeneration |
3 (MLAAttentionSpec head 512, MambaSpec, MLAAttentionSpec head 257) |
the MODEL-MM-glm5-next row's own W5 |
KV-DSV4-MULTICACHE W5 is scoped in .agents/specs/kv-dsv4-multicache.md ## Work breakdown as the DeepSeek-V4 DSA-sparse path that removes deepseek_v4.cpp's (void)attn_kv. It does not own either other forward.
The cost is measured, not hypothetical. #2343 drove GLM-5.3-Flash on dgx:gpu0 on 2026-08-30 and hit this guard on the first step. Its index row, docs/FEATURES.md, docs/USAGE.md and .agents/claims/CLAIM-GLM53-FLASH-W5B2B.md each had to reconstruct in prose what the message should have said itself: that the guard is the engine's, that it fires before dispatch to the model's own hook, and that the consuming forward is owed by the model's row rather than by the engine row.
The message also never names the arriving architecture, which is the one discriminator that would route the reader to the right row. It is free: ModelRegistry::Forward already holds LoadedModel& model, and model.registration().architecture is the registered string.
The repair
Name the architecture, computed at run time, and say that the consuming forward is owed by the row that ports that architecture. Do not enumerate the three rows in the string: an enumeration is what #2288 has already gone stale six times over on the sibling row, and a run-time architecture name cannot rot.
Out of scope
Lifting the guard. All three arriving architectures have forwards that would discard the caches (deepseek_v4.cpp:3033-3034,3105-3106 and glm5_next_registry.cpp:156 are literal (void)input.attn_kv;; qwen4_exp_registry.cpp:142 refuses unconditionally), so lifting it trades a refusal for a silent full-prefix recompute. That is a separate decision and a separate wave.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at src/vllm/model_executor/models/model_registry.cpp:462-478 and inspect how ModelRegistry::Forward accesses model.registration().architecture. Update only the multi_kv refusal wording so it names the arriving architecture and directs ownership to that architecture's model row; preserve the refusal and do not lift the guard.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai-infra-agents
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100