gpt-oss-20b-int4-ov (OpenVINO): reasoning parser not applied — harmony "analysis" channel leaks into content, chat inference fails
- Dominant language
- TypeScript
- Stars
- 979
- Forks
- 132
- Avg merge
- 9h 23m
- Merged PRs (30d)
- 8
Description
Describe the bug
The preset model OpenVINO/gpt-oss-20b-int4-ov loads correctly on the GPU, but inference in the chat UI fails / returns malformed output. The model's harmony-format output (analysis/final channels) is not parsed: the reasoning from the "analysis" channel leaks into the `content` field instead of being separated.
Environment
- OS: Windows 11
- GPU: Intel Arc A770 16GB (discrete)
- CPU: Intel Core i7-9700
- AI Playground version: 3.1.2
- Bundled OVMS: 2026.3.0
Root cause
OVMS requires BOTH `--tool_parser gptoss` AND `--reasoning_parser gptoss` for this model (per the official OpenVINO/gpt-oss-20b-int4-ov model card and the OVMS LLM docs). The models.json entry sets `"toolParser": "gptoss"` but does not set a reasoning parser, and the graph.pbtxt generated for the model contains no `reasoning_parser`. As a result, the "analysis" channel is never extracted.
Evidence
1) Querying the OVMS instance started by AI Playground via REST returns `content` beginning with "analysis..." and `finish_reason: "length"` (reasoning dumped into content).
2) Running OVMS manually with the two parsers added:
ovms.exe --model_name gptoss --model_path --tool_parser gptoss --reasoning_parser gptoss --target_device GPU --task text_generation --rest_port 8000
produces correct output: clean `content`, reasoning moved to `reasoning_content`, and `finish_reason: "stop"`.
Workarounds attempted (and why they fail)
- Editing the model's graph.pbtxt to add `reasoning_parser: "gptoss"`: AI Playground regenerates/overwrites graph.pbtxt on model load, so the change does not persist.
- Making graph.pbtxt read-only to prevent overwrite: OVMS then crashes on startup with exit code 1.
Steps to reproduce
1. Download the OpenVINO/gpt-oss-20b-int4-ov preset.
2. Select it, target device GPU.
3. Send any prompt in the chat.
4. Observe the error / malformed output containing the raw "analysis" channel.
Suggested fix
Pass `--reasoning_parser gptoss` (in addition to the existing tool parser) when launching OVMS for gpt-oss, i.e. include `reasoning_parser: "gptoss"` in the generated graph configuration. More generally, support a reasoning-parser field in models.json alongside `toolParser`, so reasoning models are handled correctly.
Contributor guide
Research direction
Start with the OpenVINO/gpt-oss-20b-int4-ov entry in models.json and trace the code that generates graph.pbtxt and launches OVMS. Confirm how toolParser is passed, then verify that reasoning_parser: "gptoss" is included and that chat responses separate reasoning_content from content without malformed output.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100