fix(TOOLS-PARSER-BREADTH): the HY-V3 streaming parser drops leading content and every parallel call after the first
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: TOOLS-PARSER-BREADTH (.agents/engine-matrix.md:166). hy_v3 is not named in the row's prose, but the row covers "the remaining --tool-call-parser dialects" and the parser-name set that src/vllm/entrypoints/openai/tool_parsers/abstract.cpp:296 enumerates, of which hy_v3 is one. Row resolved by line number before filing.
Found by wave PORTQ-7 (#2717) re-deriving PORT-NOW entry [258], upstream f18e29834b vllm#53965. Nothing was executed.
The defect
Two things are silently lost by the HY-V3 streaming tool parser:
- Content preceding
<tool_calls>in a delta is dropped whenever the buffer does not yet yield a complete tool call. - A second parallel tool call delivered in the same streaming delta is never recovered — only the first is emitted.
This tree carries the parser and carries the exact pre-commit shape at all five sites the commit touches. src/vllm/entrypoints/openai/tool_parsers/hy_v3.cpp is 625 lines, with include/vllm/entrypoints/openai/tool_parsers/hy_v3.h, registered at abstract.cpp:189, listed at :296, auto-detected at detect.cpp:101 ({"hy_v3", "<tool_calls"}), and tested through tests/vllm/entrypoints/openai/tool_parsers/test_hy_v3.cpp, wired at tests/CMakeLists.txt:2243.
The five sites:
hy_v3.cpp:386-391— the early return the commit deletes:if (!before.empty()) { DeltaMessage msg; msg.content = before; return msg; }, instead of capturingbeforeas acontent_deltacarried forward.hy_v3.cpp:407-410—if (start_idx == npos && !streaming_tool_name_) { buffer_.clear(); return std::nullopt; }, with no content emission before the drop.hy_v3.cpp:416-418—if (sep_idx == npos) { buffer_ = cur_text.substr(start_idx); return std::nullopt; }, same.hy_v3.cpp:427-434— buildsname_deltawith no.contentassignment.hy_v3.cpp:594-622— the tail ofExtractStreamingIncrementalcarries the three pre-commit branches and no_drain_callsequivalent, so nothing ever pulls a second complete<tool_call>...</tool_call>span out ofbuffer_.
grep -rni "drain" over hy_v3.cpp, hy_v3.h and test_hy_v3.cpp returns rc 1 with no hits; the positive control buffer_ over hy_v3.cpp through the identical form returns 11. grep -rni "content_delta" src/ include/ tests/ returns 10 hits, none of them in any hy_v3 file — which is its own positive control: the needle exists in this tree, just not in this parser.
The commit is the whole distance
git show 5559679229:vllm/tool_parsers/hy_v3_tool_parser.py:421-455 matches this tree line-for-line in structure — the same early return, the same two bare return Nones. Not a pre-pin hole. (hy_v3.h:1 cites "@ e24d1b24" rather than the pin, but the pin blob and this tree agree on every line the commit touches, so the older anchor does not change the reading.)
git log --oneline 5559679229..e126687a9a -- vllm/tool_parsers/hy_v3_tool_parser.py returns exactly one SHA, this one. Nothing in range narrows or withdraws it.
What is missing
Thread an std::optional<std::string> content_delta from extract_tool_calls_streaming (hy_v3.cpp:368) through ExtractStreamingIncremental (:446), emit it at the two early-out points, and hang it on name_delta and on the combined name+args delta. Add a DrainCalls(request) helper that repeatedly pulls complete <tool_call>...</tool_call> spans out of buffer_, appends them to prev_tool_call_arr / streamed_args_for_tool, and extends the returned delta's tool_calls.
Note the signature change: ExtractStreamingIncremental currently takes only (name_delta, request), so the content delta has to arrive as a parameter or a member.
Size
~60-85 product lines in one file — DrainCalls is roughly 30 in C++, plus ~12 threading edits across the five sites, plus the signature change. Tests must be authored, not ported: git show --stat f18e29834b lists one file and no test. Budget ~100-150 lines in test_hy_v3.cpp for (a) content preceding <tool_calls> in the same delta as an incomplete call, and (b) two complete calls in one delta.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/vllm/entrypoints/openai/tool_parsers/hy_v3.cpp, especially ExtractStreamingIncremental and the five listed sites, then review tests/vllm/entrypoints/openai/tool_parsers/test_hy_v3.cpp and its tests/CMakeLists.txt wiring. Run the existing HY-V3 parser tests first; done means preserving leading content and emitting all complete parallel calls, with regression tests for both cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- api, backend, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100