mudler / mudler/vllm.cpp

port(TOOLS-STREAMING-PARSER): Mistral is not on the parser-engine seam and keeps a second hand-written implementation

Open
#2,638 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: TOOLS-STREAMING-PARSER

Found by re-deriving upstream 1a20d23dab (vllm#48947) against this tree in
#2632, wave PORTQ-1.

The gap

The unified parser-engine seam exists here and carries ten model families.
Mistral is not one of them, and is still served by the hand-written parsers this
tree ported from an older revision.

src/vllm/parser/engine/registry.cpp:11-23,32-34 registers exactly qwen3,
seed_oss, kimi_k2, minimax_m2, glm47_moe, deepseek_v4, deepseek_v32,
nemotron_v3, gemma4, inkling. There is no mistral anywhere under
src/vllm/parser/ or include/vllm/parser/.

Mistral's live path is the legacy pair, both anchored at e24d1b24:

  • include/vllm/entrypoints/openai/reasoning_parsers/mistral.h:19
    class MistralReasoningParser final : public BaseThinkingReasoningParser
  • include/vllm/entrypoints/openai/tool_parsers/mistral.h:1

Upstream

1a20d23dab deletes the hand-written tool_parsers/mistral_tool_parser.py
(-782) and reasoning/mistral_reasoning_parser.py (-160), grows
vllm/parser/mistral.py into a full engine parser (+1000), registers
MistralParserReasoningAdapter / MistralParserToolAdapter in
registered_adapters.py:73-76, drops the is_mistral_tool_parser special case
from parser_manager.py:112-125 and the MistralToolParser.model_can_reason
hack from chat_completion/serving.py:159-166, and renames
_grammar_from_tool_parser to _grammar_from_parser (serving.py:343).
All references are at 1a20d23dab, a forward reference past the pin
5559679229.

Size

Large. The pin-era vllm/parser/mistral.py was never ported here, so the
work is that file plus this commit's delta: roughly 900-1200 lines — a
mistral_config() in src/vllm/parser/engine/configs.cpp carrying the
[TOOL_CALLS] / [ARGS] / [THINK] token-id grammar, its registry entry, two
adapter registrations, and retirement of the two legacy parsers. The
grammar-from-parser path (model_can_reason, _grammar_from_parser) does not
exist here either and is part of the same port.

This is a port, not a defect: Mistral tool and reasoning parsing works today
through the legacy path. What it costs is a second parser implementation kept
alive beside the seam that was built to replace it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/vllm/parser/engine/registry.cpp and the legacy headers include/vllm/entrypoints/openai/reasoning_parsers/mistral.h and include/vllm/entrypoints/openai/tool_parsers/mistral.h. Compare the port against upstream commit 1a20d23dab, then inspect configs.cpp and the parser-manager and serving entry points named in the issue. Done means Mistral uses the engine seam, its grammar and adapters are registered, and both legacy implementations and special cases are retired.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, backend
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.