Four models bypass the typed HfConfig via cfg.raw, so normalization does not reach them
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: ENG-ATTENTION-WINDOW
What
Four models read a config value by falling back to the raw checkpoint
document, which defeats every normalisation LoadHfConfig performs on the
typed field:
g.sliding_window = cfg.sliding_window.value_or(RawInt(cfg.raw, "sliding_window", 0));
src/vllm/model_executor/models/gemma2.cpp:103src/vllm/model_executor/models/gemma3.cpp:101src/vllm/model_executor/models/gemma4.cpp:139src/vllm/model_executor/models/laguna_weights.cpp:117
cfg.raw is the full untouched JSON (hf_config.cpp:598, nlohmann::json raw; // full doc for fields we don't type yet). So when the typed field is
std::nullopt, these four do not get "absent" — they get whatever the checkpoint
literally said.
Why it matters, with the case that found it
HfConfig normalises a checkpoint sliding_window == 0 to std::nullopt
(hf_config.cpp:80-87), mirroring ModelConfig's own conversion. At these four
sites that normalisation is undone: value_or falls through to
RawInt(raw, "sliding_window", 0) and reads the 0 straight back.
That specific case is currently benign by luck — the call sites guard
*sliding_window > 0, so a 0 is inert either way.
It is not benign for the mechanism upstream actually uses to disable the
window. vllm/config/model.py:766-769 implements
ModelConfig.disable_sliding_window by setting hf_text_config.sliding_window = None. Transcribed here, that would null the typed field and these four models
would keep their window from cfg.raw, silently. ENG-ATTENTION-WINDOW W3 hit
this and had to consume its switch at the point of USE
(ResolveAttentionWindow) instead of at the point of configuration.
So the bug is not "a wrong window today". It is that the typed config is not
authoritative, and any future normalisation, override, or CLI flag that works
by writing the typed field will appear to work and will not.
Scope
sliding_window is the instance that was found. The value_or(RawInt(raw, ...))
shape should be swept for — it is a general pattern, not a one-off, and each
instance is a place a normalised field can be bypassed.
Not fixed in flow
Repointing four models off cfg.raw changes what each reads when a checkpoint
and the typed parse disagree, which is a behaviour change per model with its own
gate. It is filed rather than folded into W3, whose switch is correct without it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the four call sites in src/vllm/model_executor/models/gemma2.cpp, gemma3.cpp, gemma4.cpp, and laguna_weights.cpp, then inspect hf_config.cpp:80-87 and :598 to understand typed normalization and cfg.raw. Sweep for the same value_or(RawInt(raw, ...)) pattern; done means every relevant normalized field remains authoritative without folding this work into W3.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100