[Bug]: NemotronV3 reasoning parser surfaces a trailing think tag as content when the stream ends on it
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
- CPU architecture: N/A (reproducible with a pure-Python snippet, no GPU required)
- GPU: N/A
- TensorRT-LLM branch:
main - TensorRT-LLM commit:
f7f596b94e - OS: N/A
The report concerns NemotronV3ReasoningParser in tensorrt_llm/llmapi/reasoning_parser.py
and affects streaming /v1/chat/completions responses for reasoning_parser: nano-v3.
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
DeepSeekR1Parser.parse_delta holds back a delta that is exactly a <think> or
</think> tag (reasoning_end.startswith(delta_text) is true for equality) and waits for
the next delta to decide what it is. When the stream ends on that delta, finish() is
responsible for the withheld buffer. DeepSeekR1Parser.finish() discards a buffer that
holds exactly a tag (reasoning_parser.py:316-320, added in #17157).
NemotronV3ReasoningParser.finish() (reasoning_parser.py:627-655) reimplements the
method without that rule, so the tag is surfaced as model output:
from tensorrt_llm.llmapi.reasoning_parser import ReasoningParserFactory
for kwargs in (None, {"force_nonempty_content": True}):
parser = ReasoningParserFactory.create_reasoning_parser("nano-v3", kwargs)
for delta in ["a", "</think>"]:
parser.parse_delta(delta)
print(kwargs, parser.finish())
parser = ReasoningParserFactory.create_reasoning_parser("nano-v3", {"enable_thinking": False})
parser.parse_delta("<think>")
print(parser.finish())
Output on main:
None ReasoningParserResult(content='', reasoning_content='</think>')
{'force_nonempty_content': True} ReasoningParserResult(content='a</think>', reasoning_content='')
ReasoningParserResult(content='<think>', reasoning_content='')
</think> is a single token for this model, so any generation that closes its reasoning
and then stops (an empty answer, or a stop right after the tag) ends the stream on exactly
that delta and hits the first two cases on every such request.
Expected behavior
A stream that ends on a complete tag yields the same split as parse() on the joined
text: parse("a</think>") gives reasoning_content="a", content="" (or content="a"
with force_nonempty_content), and parse("<think>") with thinking disabled gives empty
content. The tag is a delimiter and is never part of reasoning_content or content.
actual behavior
The literal </think> is appended to reasoning_content (or, with
force_nonempty_content, to content), and a literal <think> is returned as content
when thinking is disabled. Streaming and non-streaming responses differ for the same
generation.
additional notes
DeepSeekR1Parser.finish()is the in-package precedent for the rule; the Nemotron
override predates it and reimplements the parent's flush without it.- The existing
test_nano_v3_reasoning_parser_finishcovers</think>first and in the
middle of the stream, but never as the last delta. - I have a fix ready (two lines in
NemotronV3ReasoningParser.finish(), plus three
parametrizations in that test) and will open a PR referencing this issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tensorrt_llm/llmapi/reasoning_parser.py, comparing NemotronV3ReasoningParser.finish() with DeepSeekR1Parser.finish(). Run the existing test_nano_v3_reasoning_parser_finish and extend its cases for a stream ending on a complete tag; done means streaming finish() matches parse() without returning delimiter tags as content or reasoning content.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100