Truncated tool call is emitted as a complete one (unterminated heredoc reaches the client)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 152
- Avg merge
- 4h 14m
- Merged PRs (30d)
- 11
Description
Summary
When generation stops inside a <tool_call> block — normally because it hit the
output limit while writing a large argument — the parser takes the remainder of
the text as the block and emits a complete-looking tool call built from partial
input. The client cannot detect this: the arguments JSON is well formed,
because it is rebuilt from the parsed parameters. A shell heredoc cut mid-body
becomes an unterminated cat <<EOF.
Affects qwen3_5vl, qwen3_6_moe and qwen3_5_omni. Reproduced on main at
92f3f13.
Environment
- OS: Ubuntu 26.04.1 LTS (kernel 7.0.0-31-generic)
- NPU driver: NPU FW 1.1.2.64, amdxdna 0.7
flm version:FLM v1.0.6flm validate:
[Linux] Kernel: 7.0.0-31-generic
[Linux] NPU: /dev/accel/accel0 with 8 columns
[Linux] NPU FW Version: 1.1.2.64
[Linux] amdxdna version: 0.7
[Linux] Memlock Limit: infinity
Reproduce
flm serve qwen3.5:9b
curl -s localhost:52625/v1/chat/completions -H 'Content-Type: application/json' -d '{
"model": "qwen3.5:9b",
"messages": [{"role":"user","content":"Use the bash tool to write a very long 300-line poem to /tmp/p.txt using a cat heredoc."}],
"max_tokens": 400,
"tools": [{"type":"function","function":{"name":"bash","description":"Run a shell command",
"parameters":{"type":"object","properties":{"command":{"type":"string"}},"required":["command"]}}}],
"tool_choice": "auto"
}' | jq '.choices[0]'
Observed on 92f3f13:
finish_reason : length
tool_calls : True
args length : 1580
args tail : '...n a sacred soundless gown.\n\nSo let us walk gently on this precious"}'
The heredoc has no closing EOF. A client that executes this hangs waiting for
the terminator.
Cause
parse_nstream_content, e.g. modeling_qwen3_5vl.cpp:493-500:
} else {
// Unclosed tag — search for </function> fallback
size_t func_end_pos = response_text.find(func_end_tag, block_content_start);
if (func_end_pos != std::string::npos) {
block_end = func_end_pos + func_end_tag.length();
} else {
block_end = response_text.length(); // <-- fabricates a complete call
}
The </function> fallback is reasonable: a block missing only the outer
</tool_call> is still complete. The final else is not — neither closing tag
present means the call was never finished.
Impact
finish_reason is length, which is the only hint, and clients commonly ignore
it when tool_calls is present. The result is a syntactically valid tool call
carrying a truncated command. For file-writing agents that is an unterminated
heredoc; for other tools it is a silently truncated argument.
Fix
PR follows: drop the block when neither closing tag is present, so no tool call
is emitted. The response still carries finish_reason: "length", which is the
honest signal that the client should retry with a larger budget. Complete calls
and the </function> fallback are unaffected.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in modeling_qwen3_5vl.cpp around lines 493-500 and trace parse_nstream_content for blocks lacking both closing tags. Use the provided flm serve and curl reproduction to check the emitted tool_calls and finish_reason. Done means incomplete tool-call blocks are not emitted, while complete calls and the fallback still work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100