fix(TOOLS-STREAMING-PARSER): bare MESSAGE_HEADER text is dropped instead of emitted
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: TOOLS-STREAMING-PARSER
Found by re-deriving upstream 629a938a92 (vllm#50403) against this tree in
#2632, wave PORTQ-1.
The defect
Text a model emits while the streaming parser is in MESSAGE_HEADER is dropped
and never reaches the client. The engine has no arm for that state and no buffer
to hold it, so the text is discarded at four separate points:
src/vllm/parser/engine/streaming_parser_engine.cpp:297-305—emit_for_state
has noMESSAGE_HEADERarm; it falls through tocontent_event_forand
returns{}when that misses.include/vllm/parser/engine/parser_engine_config.h:73-78— thecontent_events
default map carriesCONTENT/REASONING/TOOL_NAME/TOOL_ARGSonly, so
MESSAGE_HEADERalways misses.src/vllm/parser/engine/streaming_parser_engine.cpp:221-222—finish()
leavesMESSAGE_HEADERby settingstate_ = CONTENTand emitting nothing.src/vllm/parser/engine/configs.cpp:1002—
c.transitions[{S::MESSAGE_HEADER, end}] = Transition(S::CONTENT, {});
(and:985,:990,:991for the otherMESSAGE_HEADERexits), the empty
event tuple upstream's fix changes.
grep -rn 'message_header_buffer\|MessageHeaderBuffer' include src returns
nothing.
Upstream
vllm/parser/engine/streaming_parser_engine.py @ 629a938a92 (a forward
reference past the pin 5559679229): :359-361 buffers MESSAGE_HEADER text
into _message_header_buffer instead of dropping it, :388-425 emits the buffer
as a TEXT_CHUNK on the transition out and suppresses empty chunks, :274-282
re-emits it when finish() leaves MESSAGE_HEADER, and :188/:324 clear it on
reset and on the skip-tool-parsing path. parser/inkling.py:270-271 changes the
(MESSAGE_HEADER, end) transition's events from () to (TEXT_CHUNK,).
This is user-visible: a bare Inkling response with no thinking block loses its
text.
Size
~35-45 lines in src/vllm/parser/engine/streaming_parser_engine.cpp, one member
in its header, one line in src/vllm/parser/engine/configs.cpp, plus a red-first
streaming test.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with emit_for_state and finish() in src/vllm/parser/engine/streaming_parser_engine.cpp, then inspect the content_events defaults in include/vllm/parser/engine/parser_engine_config.h and MESSAGE_HEADER transitions in src/vllm/parser/engine/configs.cpp. Compare the referenced upstream behavior and add the red-first streaming test mentioned in the issue; done means bare MESSAGE_HEADER text reaches the client, including on finish, without empty chunks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, stream-processing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100