mudler / mudler/vllm.cpp

fix(MODEL-MM-gemma4-mm-gemma4-for-conditional-generation): the Gemma-4 MM path applies no suppress_tokens logit mask, a pre-pin hole

Open
#2,697 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: MODEL-MM-gemma4-mm-gemma4-for-conditional-generation

Found by wave PORTQ-4 (#2680)
while re-deriving PORT-NOW entry 148 of 5559679229..e126687a9a, upstream
f97e502969 vllm#51738.

This is not that commit's content. f97e502969 only changes how the mask is
applied (list-index masking to a cached index tensor plus index_fill_), and entry
148 is classified NOT_APPLICABLE for that reason. What the classification exposed
is that the mask itself is missing, and has been since before the pin.

What is wrong

This tree's Gemma-4 multimodal path applies no suppress_tokens logit mask at
all
. src/vllm/model_executor/models/gemma4_mm.cpp is 298 lines; it computes
logits at :248 and does nothing to them but argmax and diagnostics (:76-98).

Upstream has had the behaviour since before the pin:
git show 5559679229:vllm/model_executor/models/gemma4_mm.py reads suppress_tokens
from the generation config at :1147 and masks those ids to -inf at :2038-2039.

Searches behind the claim of absence

  • Search 1, grep -rniF -e suppress_tokens -e generation_config over src/ and
    include/: generation_config hits only
    src/vllm/v1/engine/input_processor.cpp:39-75 (EOS ids); suppress_tokens is
    0 hits
    .
  • Search 2, different spelling and location, grep -rniF -e suppress -e "-inf" -e infinity over src/vllm/model_executor/models/gemma4*.cpp: one unrelated hit,
    gemma4_vision.cpp:165, an activation clamp.
  • Positive control through the identical probe form: grep -rniF -e eos_token_id
    returns 171 hits.
  • Scope: src/ and include/ of the worktree at a700e8da6.

Why a token gate would not have caught it

The divergence only appears on a checkpoint whose generation_config.json carries a
non-empty suppress_tokens. Whether any Gemma-4 checkpoint this tree gates
against does is exactly what could not be determined
— reading a
generation_config.json was out of scope for a read-only pass, and nothing in this
wave was executed. That question decides whether this hole is live or latent, and it
should be settled before the fix is sized.

Adjacent record note

.agents/sync/2026-09-01-adebc41.md:649 classified the sibling Gemma-4 MTP
suppress-tokens commit (vllm#53884) as IGNORE because "gemma4_mtp is unported". That
reading is correct for the MTP head and did not notice that the base masking is
unported too.

Verification state

Nothing was executed — no build, no test run, no GPU, no lease. Every
path:line was printed from the tree at a700e8da6. ALREADY_SATISFIED /
REAL_GAP here means the code does or does not implement the behaviour, not that a
gate observed it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/vllm/model_executor/models/gemma4_mm.cpp around the logits computation at line 248, then inspect src/vllm/v1/engine/input_processor.cpp for existing generation-config handling. Check the relevant checkpoint's generation_config.json to establish whether suppress_tokens is present, and verify that the Gemma-4 multimodal path honors it without changing unrelated diagnostics or argmax behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.