[Bug]: standalone Responses API input, streaming, and tool continuation failures with Agentic API
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
hello, i'm Francisco, maintainer of vLLM Agentic API. i'm testing streaming, conversation continuation, and tool calls with Qwen3-8B and GPT-OSS-20B against TensorRT-LLM's standalone /v1/responses. our Dynamo → TensorRT-LLM integration worked, but the standalone path exposed these issues:
- input conversion: typed messages with string content become empty on the Qwen path; empty strings raise an error.
- output replay: returned
logprobs:nullcan fail input validation. summary-only, empty, or multipart reasoning items can also fail or lose content. - streaming: item IDs, call IDs, and output indices can disagree with the terminal response. Harmony tool-call events are missing, and incomplete responses get a
response.completedevent. - continuation: streamed responses aren't saved by the frontend path even with storage enabled.
previous_response_idwithstore:falsealso drops prior history. - tool results: stored chat history can lose tool calls; GPT-OSS/Harmony misses some call channels/recipients and fails to pair replayed calls with their results.
- instructions: Harmony continuation reuses old instructions and tool definitions instead of the current request's.
i have local fixes for these. the tests pass and through it works using vLLM Agentic API on an NVIDIA GB10, using TensorRT-LLM 1.3.0rc26 with patched serving code, OpenAI SDK 3.8.0, and Pydantic 2.13.4. this tested one frontend with frontend postprocessing, not a full rebuild of main.
the first draft, #19222, covers input conversion/replay and includes regression tests: 86 unit tests and pre-commit checks passed. the live results above cover the combined local fixes, not that PR alone. i'll keep streaming/storage and tool-history/Harmony fixes in separate PRs.
does this split make sense, and are any of these behaviors intentional restrictions of the standalone API?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing draft #19222 and its regression tests for input conversion and replay, then compare standalone /v1/responses behavior with the listed streaming, storage, tool-history, and Harmony cases. The remaining work is planned as separate pull requests; done means each behavior has agreed scope and corresponding regression coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, api, backend
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100