vllm-project / vllm-project/vllm
[Bug]: RustToolParser streaming can drop buffered text at stream end
- Dominant language
- Python
- Stars
- 91.9k
- Forks
- 22.3k
- PR merge metrics
- PR metrics pending
Description
## Your current environment
- vLLM `main` at `04d13b5`
- Reproduced with the Rust tool-parser Python extension in the `vllm/vllm-openai:latest` container
- CPU-only parser reproduction; no model weights or GPU execution are required
## Describe the bug
`RustToolParser` does not finalize the underlying Rust parser when streaming generation ends. A Rust parser may retain an ambiguous partial tool-call marker in its incremental buffer, so the buffered suffix can be silently omitted from streamed content.
For MiniMax M3, the minimal model output is:
```text
plain ]<]minimax[>[[
Contributor guide
Research direction
Start with the streaming adapter's TODO about calling Rust finish(), then trace how DelegatingParser reports finished=True and compare it with the non-streaming adapter's parse_into and finish sequence. Reproduce the CPU-only MiniMax M3 string and verify that stream-end output preserves the buffered suffix, including recovery through reset() when finalization fails for incomplete syntax.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100