mudler / mudler/vllm.cpp

fix(engine): the engine never range-checks prompt token ids, and the upper bound was already owed at the pin -- no row owns it

Open
#2,699 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: none owns this. No matrix or roadmap row names the InputProcessor vocab check. The obligation is recorded at .agents/porting-inventory.md:52 ("Encoder arm + out-of-vocab check still deferred") and in the header itself (include/vllm/v1/engine/input_processor.h:44,50, both listing it under DEFERRED). The nearest sibling row is SERVE-REQUEST-LENGTH-GUARD (.agents/engine-matrix.md:250), which owns the other arm of the same upstream function (_validate_prompt_len) but is scoped to a byte bound. Stating this plainly rather than inventing an owner; a row is owed.

Found by wave PORTQ-5 (#2679) re-deriving PORT-NOW entry [164], upstream b216db3ed0 vllm#51795. Nothing was executed.

The gap is bigger than the commit

Upstream's commit adds only the negative arm: min_input_id = min(prompt_ids) and a VLLMValidationError beside the pre-existing upper-bound check in InputProcessor._validate_model_input.

Neither bound exists here. src/vllm/v1/engine/input_processor.cpp:286-317 (process_inputs_tokens) validates only ValidateParams(params) and ValidatePromptLen(prompt_token_ids.size()) and then moves the ids straight into the request at :312. process_inputs (:245-284) and process_inputs_mm (:319-350) are the same three steps. ValidatePromptLen (:85-110) checks only emptiness and len > max_model_len_.

The only token_id < 0 test in the file is at :228-238, and it is the bad_words n-gram range check (a port of sampling_params.py:683-698) — a different input entirely.

The upper bound is a pre-pin hole

git show 5559679229:vllm/v1/engine/input_processor.py already carries:

if max_input_id > max(tokenizer.max_token_id, model_vocab_size - 1):
    raise ValueError(f"Token id {max_input_id} is out of vocabulary")

So the distance from upstream is the whole check, not this commit. A commit-range queue structurally cannot see that, and porting only the negative arm would leave the upper bound still missing.

Consequence

A negative id reaches the embedding index. Both generate and the pooling/embedding paths share the entry point.

Size

~20-30 lines. The loop is short, but InputProcessor deliberately does not retain HfConfig (include/vllm/v1/engine/input_processor.h:80-84: "the HfConfig need not [outlive] — it is fully consumed here"), so the model vocab size needs the same derive-up-front plumbing max_model_len_ got, in order to mirror max(tokenizer.max_token_id, model_vocab_size - 1) rather than tokenizer_.VocabSize() alone.

Positive control for the absence claim (identical probe form, scope src/+include/+tests/): grep -rnF "out of vocabulary" returns 3 hits, all in model_executor/models/minimax_h3_encoder*, none in the engine; grep -rniF "vocab" over the two InputProcessor files returns 8 hits, all the logprobs cap or the bad_words check.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/vllm/v1/engine/input_processor.cpp:245-350 and include/vllm/v1/engine/input_processor.h:44,50,80-84; compare the deferred notes with the existing ValidatePromptLen flow. Trace how max_model_len_ is derived, then inspect the three process_inputs entry points. Done means both negative and upper prompt-token-id bounds are enforced for generate and pooling/embedding paths, with the required vocabulary-size plumbing in place.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.