port(SERVE-REQUEST-LENGTH-GUARD): four generation-input bounds are missing, and beam_width bypasses the one that exists
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: SERVE-REQUEST-LENGTH-GUARD
Found by wave PORTQ-4 (#2680)
re-deriving PORT-NOW entry 137 of 5559679229..e126687a9a, upstream
b64a2708b0 vllm#51447
"Bound generation inputs before expensive work".
That title is this row's own thesis, which is why the row owns it.
Four bounds are missing, and one of them is reachable today
beam_width is entirely unvalidated, and the existing n bound does not cover
it. BeamSearchParams is a plain aggregate with no validation
(include/vllm/entrypoints/beam_search.h:61-73), and beam_width is assigned
straight from the request n at
src/vllm/entrypoints/openai/protocol.cpp:741 (bp.beam_width = n;) and :752
(bp.beam_width = n.value_or(1);). The beam branch at
src/vllm/entrypoints/openai/serving_completion.cpp:222-241 calls
to_beam_search_params directly and bypasses to_sampling_params, so
SamplingParams::Verify's n > kMaxNSequences check
(src/vllm/sampling_params.cpp:48-55, kMaxNSequences = 16384 at
include/vllm/sampling_params.h:60) never runs on this path. An unvalidated n
reaches BeamSearchDrive.
The other three:
- No dedup of
stop_token_idsorbad_words.PostInit
(src/vllm/sampling_params.cpp:243-257) builds a deduplicatingstd::setfor the
derivedall_stop_token_ids, but leaves the source vectors as given. - No stop-string count bound.
ParseStop
(src/vllm/entrypoints/openai/protocol.cpp:100-107) takes the array unbounded;
VLLM_MAX_STOP_STRINGS,max_stop_stringsandkMaxStopStringsare 0 hits
acrosssrc/,include/andtests/, against a positive control of
VLLM_MAX_N_SEQUENCESwhich is found atsampling_params.cpp:55and
sampling_params.h:59. - No bad-word count cap.
src/vllm/sampling_params.cpp:165-176validates only
that no bad word is the empty string.
What is not portable
Upstream's v1/worker/gpu/sample/bad_words.py half is surface-absent. Its
MAX_BAD_WORDS_TOTAL_TOKENS / MAX_NUM_BAD_WORDS exist to size two fixed-width
GPU staging buffers. This tree has none: bad words are a dynamic nested container,
std::map<int, std::vector<std::vector<int32_t>>> bad_words_token_ids at
include/vllm/v1/sample/metadata.h:84, filled per slot at
src/vllm/v1/worker/gpu/input_batch.cpp:356-360 and read by
include/vllm/v1/sample/ops/bad_words.h:22. There is nothing to size. The
validation remains portable and is where the bound belongs here.
Upstream's envs.py half also differs in mechanism: this tree expresses such bounds
as compile-time constants (include/vllm/sampling_params.h:60), so the port is
three constants beside kMaxNSequences, not an env table.
Size
Roughly 40-70 lines: two dedups, one beam_width verify, one stop-count check in
both from_jsons, one bad-word count check, three constants, and tests.
Co-owning rows for the sampling-side halves: SAMPLE-BEAM
(.agents/engine-matrix.md:147) for beam_width, SAMPLE-LOGIT-FILTERS (:145)
for the dedup and bad-word cap — its anchors already name sampling_params.cpp
validation and input_processor.cpp bad-words tokenization.
A decision this needs, not just a port
Mirroring VLLM_MAX_STOP_STRINGS = 4 is a new, tight, user-visible refusal. It
may refuse requests this server serves today. Policy says mirror vLLM; the point is
that the refusal should be a stated decision in whichever spec claims this, not a
silent consequence.
Verification state
Nothing was executed — no build, no test run, no GPU, no lease. Every
path:line was printed from the tree at a700e8da6; the beam_width bypass chain
was re-read by the wave operator. b64a2708b0 is not an ancestor of the pin, and
the n bound it refactors predates the pin and is already mirrored here — which is
the correct pre-pin reading. No prior triage: git log --all --grep b64a2708b0 is
empty.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with include/vllm/sampling_params.h and src/vllm/sampling_params.cpp, then trace request parsing in src/vllm/entrypoints/openai/protocol.cpp and the beam branch in src/vllm/entrypoints/openai/serving_completion.cpp. Review existing sampling tests and add coverage for beam_width, duplicate inputs, stop-string counts, and bad-word counts. Done means all four bounds are enforced and the stop-string limit is explicitly decided and documented in the relevant spec.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100