MODEL_TYPE_TO_TOOL_PARSER maps qwen3_5/qwen3_5_moe to a parser whose format the template does not emit
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
- TensorRT-LLM: 1.3.0rc23
- GPU: NVIDIA GeForce RTX 5090 (sm_120), driver 595.71.05
- Backend: PyTorch; single GPU, no TP/PP
- Models:
Qwen/Qwen3.8-27B(qwen3_5family); same class of mismatch previously observed on Qwen3.6-35B-A3B
Who can help?
No response
Reproduction
MODEL_TYPE_TO_TOOL_PARSER (tensorrt_llm/serve/tool_parser/tool_parser_factory.py) maps model
types qwen3_5 and qwen3_5_moe to the qwen3 parser, which expects the JSON tool-call form.
The chat template for these models emits the XML-ish form instead:
<tool_call><function=name><parameter=k>v</parameter></function></tool_call>
Serve with automatic tool-parser selection and issue a request with tools defined. Measured on
a four-call scenario:
- with the auto-selected
qwen3parser: 0 of 4 tool calls parsed - with
--tool_parser qwen3_coder: 4 of 4 parsed, arguments correct including an optional enum
Expected behavior
Automatic selection maps a model family to a parser that can parse the format that family's template emits.
actual behavior
tool_calls comes back [] and content is emptied, with HTTP 200 and no error or warning. The response is indistinguishable from a model that chose not to call a tool, so a caller cannot detect the failure.
additional notes
--tool_parser qwen3_coder parses this format correctly on the same build and model, so the
parser exists and only the mapping is wrong.
Related but distinct: #17917 reports that a parser which matches its marker yet extracts no calls
returns 200 with no diagnostic — that is the silence; this issue is the mapping that triggers
it here. Fixing either alone leaves the other. #17574 and #17740 concern the streaming parser.
Questions for maintainers
- Should
qwen3_5/qwen3_5_moemap toqwen3_coder, or is the correct discriminator the
template content rather than the model type? - Are other entries in
MODEL_TYPE_TO_TOOL_PARSERverified by effect against their templates?
The failure mode here is silent, so an unverified entry looks identical to a working one.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tensorrt_llm/serve/tool_parser/tool_parser_factory.py and compare the qwen3 and qwen3_coder parser mappings with the tool-call format emitted by the qwen3_5 templates. Reproduce automatic parser selection with tools defined and verify that qwen3_5 and qwen3_5_moe produce parsed tool calls with their arguments rather than empty content and tool_calls.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 74/100