vllm-project / vllm-project/vllm
[Bug]: DSML tool-call recovery (#55954) emits ghost empty-argument calls on truncation and narration, and recovers undeclared tools
- Dominant language
- Python
- Stars
- 91.8k
- Forks
- 22.2k
- PR merge metrics
- PR metrics pending
Description
## Summary
`#55954` recovers DSML tool calls when the model omits the `<|DSML|tool_calls>` wrapper by committing as soon as it sees the `<|DSML|invoke name="` prefix. On three reachable shapes this produces a **tool call with empty arguments plus a truncated assistant message**, and on a fourth it **recovers a tool the request never declared**. `#56141` fixes only the misspelled-wrapper case; the family above is unchanged on current main.
Measured on the `deepseek_v4` parser at `e7edf17c` with a 12-case matrix (each case run both non-streaming and streaming). "ghost" = a tool call whose `arguments` is `""` or `{}`. "delivered" = characters handed to the caller vs. input length.
| case | expected | stock `e7edf17c` | with #52645 semantics |
|---|---|---|---|
| N1 narration then bare unclosed `invoke` (truncated) | plain content | `n=1 ghost=True` delivered **33/61** | `n=0` delivered 61/61 |
| N7 narration mentioning the invoke syntax (zh) | plain content | `n=1 ghost=True` delivered **5/49** | `n=0` delivered 49/49 |
| N8 narration mentioning the invoke syntax (en) | plain content | `n=1 ghost=True` delivered **52/94** | `n=0` delivered 94/94 |
| N9 orphan `invoke` naming an **undeclared** tool | plain content | **`n=1`** delivered 6/120 | `n=0` delivered 120/120 |
| R1 misspelled wrapper `toolcalls` | recover | `n=1` ✔ (this is #56141) | `n=1` ✔ |
| R2 `<|DSML|parameter>` without `string=` | recover | `n=1 ghost=True` | `n=1` ✔ |
| N2–N6, R3 | — | ✔ | ✔ |
Totals: stock **5 failures**, stock + the two small hunks (`string=` optional, `finish_reason=length`) **4 failures**, #52645 semantics **0 failures**. Streaming and non-streaming agree on every case.
An extra check: with `tools=None` / `tool_choice="none"`, stock still recovers (`n=1`, delivers 10/118 characters); the #52645 semantics deliver the full 118 characters and emit no call.
## Why this matters in practice
N1 is the common one. A request that hits `max_tokens` mid-`invoke` currently yields both a chopped assistant message and a `('bash', '{}')`-shaped call. A harness that does not inspect `finish_reason` will dispatch an argument-less tool call; one that does will drop the turn, but the visible text was already truncated.
N9 is a contract issue independent of truncation: the parser should not synthesize a call for a tool the request did not declare.
## What behaves correctly
`#52645`'s "hold provisional → validate the tool name against the request's declared tools → commit only at `` → otherwise roll back to the original text" passes all 12 cases in both modes, including N9 and the no-tools case. Its own 87-case suite also passes unmodified.
## Suggestion
Gate the recovery in #55954 on (a) the tool name being present in the request's declared tools, and (b) the invoke block actually closing, rolling the buffered text back verbatim when either fails — i.e. adopt #52645's provisional-commit discipline rather than committing on the opening prefix.
Happy to share the matrix script; it is self-contained and needs only the parser (no GPU).
## Environment
vLLM `e7edf17c` (`0.28.1rc1.dev681`), `deepseek_v4` tool parser, DeepSeek-V4-Flash. Matrix run in a CPU-only one-off container against the shipped parser.
Contributor guide
Assessment
This issue has not been assessed yet.