mudler / mudler/vllm.cpp

ModelRegistry::Forward's multi-cache refusal names ONE owner where THREE architectures now arrive, and never names the arriving architecture

Open Beginner friendly
#2,353 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

ModelRegistry::Forward's multi_kv refusal (src/vllm/model_executor/models/model_registry.cpp:462-478) ends with:

(row KV-DSV4-MULTICACHE W5 owns the consuming forward; #1925, #2068)

That clause was true when W3 landed it, because DeepSeek-V4 was the only architecture that could publish a multi-cache topology. Three architectures reach the guard now, and the clause is false for two of them.

architecture groups who owns its consuming forward
DeepseekV4ForCausalLM 7, all MLAAttentionSpec / SlidingWindowMLASpec KV-DSV4-MULTICACHE W5 -- the clause is true here
Qwen4ExpForConditionalGeneration 3 (FullAttentionSpec, MambaSpec, MLAAttentionSpec) MODEL-MM-QWEN4-EXP. Qwen4ExpTextModel::Forward does not exist
Glm5NextForConditionalGeneration 3 (MLAAttentionSpec head 512, MambaSpec, MLAAttentionSpec head 257) the MODEL-MM-glm5-next row's own W5

KV-DSV4-MULTICACHE W5 is scoped in .agents/specs/kv-dsv4-multicache.md ## Work breakdown as the DeepSeek-V4 DSA-sparse path that removes deepseek_v4.cpp's (void)attn_kv. It does not own either other forward.

The cost is measured, not hypothetical. #2343 drove GLM-5.3-Flash on dgx:gpu0 on 2026-08-30 and hit this guard on the first step. Its index row, docs/FEATURES.md, docs/USAGE.md and .agents/claims/CLAIM-GLM53-FLASH-W5B2B.md each had to reconstruct in prose what the message should have said itself: that the guard is the engine's, that it fires before dispatch to the model's own hook, and that the consuming forward is owed by the model's row rather than by the engine row.

The message also never names the arriving architecture, which is the one discriminator that would route the reader to the right row. It is free: ModelRegistry::Forward already holds LoadedModel& model, and model.registration().architecture is the registered string.

The repair

Name the architecture, computed at run time, and say that the consuming forward is owed by the row that ports that architecture. Do not enumerate the three rows in the string: an enumeration is what #2288 has already gone stale six times over on the sibling row, and a run-time architecture name cannot rot.

Out of scope

Lifting the guard. All three arriving architectures have forwards that would discard the caches (deepseek_v4.cpp:3033-3034,3105-3106 and glm5_next_registry.cpp:156 are literal (void)input.attn_kv;; qwen4_exp_registry.cpp:142 refuses unconditionally), so lifting it trades a refusal for a silent full-prefix recompute. That is a separate decision and a separate wave.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at src/vllm/model_executor/models/model_registry.cpp:462-478 and inspect how ModelRegistry::Forward accesses model.registration().architecture. Update only the multi_kv refusal wording so it names the arriving architecture and directs ownership to that architecture's model row; preserve the refusal and do not lift the guard.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai-infra-agents
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.