NVIDIA / NVIDIA/TensorRT-LLM

[Bug]: Qwen3-Coder streaming tool parser never completes a zero-argument tool call

Open
#19,227 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

System Info
  • CPU architecture: N/A (reproducible with a pure-Python snippet, no GPU required)
  • GPU: N/A
  • TensorRT-LLM branch: main
  • TensorRT-LLM commit: f7f596b94e
  • OS: N/A

The report concerns tensorrt_llm/serve/tool_parser/qwen3_coder_parser.py and affects
streaming /v1/chat/completions responses for tool_parser: qwen3_coder, which
tool_parser_factory.py also selects for nemotron_h_omni.

Who can help?

No response

Information
  • The official example scripts
  • My own modified scripts
Tasks
  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)
Reproduction

A tool call whose function takes no parameters is written by the model as a
<function=...> block with no <parameter> children. In streaming,
Qwen3CoderToolParser.parse_streaming_increment emits the name, then on </tool_call> it
only appends a closing brace when some argument text was already streamed
(qwen3_coder_parser.py:156-172):

current_streamed = self.streamed_args_for_tool[self.current_tool_id]
if current_streamed:
    ...
    if open_braces > close_braces:
        calls.append(ToolCallItem(..., parameters="}"))

With no parameter block nothing was streamed, so the branch is skipped and the call
completes with an empty arguments string. detect_and_parse on the same text returns
{}.

from tensorrt_llm.serve.openai_protocol import ChatCompletionToolsParam, FunctionDefinition
from tensorrt_llm.serve.tool_parser.qwen3_coder_parser import Qwen3CoderToolParser

tools = [ChatCompletionToolsParam(
    type="function",
    function=FunctionDefinition(
        name="get_time", description="Get current time",
        parameters={"type": "object", "properties": {}}))]

text = "<tool_call>\n<function=get_time>\n</function>\n</tool_call>"

streamed = Qwen3CoderToolParser().parse_streaming_increment(text, tools)
print([(c.name, c.parameters) for c in streamed.calls])
oneshot = Qwen3CoderToolParser().detect_and_parse(text, tools)
print([(c.name, c.parameters) for c in oneshot.calls])

Output on main:

[('get_time', '')]
[('get_time', '{}')]

The same happens when the call arrives across several deltas: the name is streamed, and
no argument delta ever follows.

Expected behavior

Concatenating the streamed argument deltas of a call yields the same JSON that
detect_and_parse returns for it. A zero-argument call streams {}, as the other
parsers in the package already do (Glm47ToolParser._finalize_tool_call sends "{}"
when no parameter was streamed, and BaseToolParser.parse_streaming_increment was fixed
to do the same in #17575).

actual behavior

The streaming client receives function.arguments == "", which is not valid JSON, so a
client that parses the arguments before invoking the tool fails on every zero-argument
call, while the non-streaming request for the same output gets {}.

additional notes
  • nemotron_h_omni is mapped to this parser in tool_parser_factory.py, so it is affected
    as well.
  • I have a fix ready (stream "{}" on completion when nothing was streamed, mirroring
    Glm47ToolParser) and will open a PR referencing this issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tensorrt_llm/serve/tool_parser/qwen3_coder_parser.py, especially Qwen3CoderToolParser.parse_streaming_increment around lines 156-172, and compare it with detect_and_parse and Glm47ToolParser._finalize_tool_call. Reproduce the zero-argument example and verify that streamed arguments concatenate to the same valid {} JSON returned by non-streaming parsing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.