intel / intel/AI-Playground

gpt-oss-20b-int4-ov (OpenVINO): reasoning parser not applied — harmony "analysis" channel leaks into content, chat inference fails

Open
#556 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
979
Forks
132
Avg merge
9h 23m
Merged PRs (30d)
8

Description

Describe the bug
The preset model OpenVINO/gpt-oss-20b-int4-ov loads correctly on the GPU, but inference in the chat UI fails / returns malformed output. The model's harmony-format output (analysis/final channels) is not parsed: the reasoning from the "analysis" channel leaks into the `content` field instead of being separated.

Environment
- OS: Windows 11
- GPU: Intel Arc A770 16GB (discrete)
- CPU: Intel Core i7-9700
- AI Playground version: 3.1.2
- Bundled OVMS: 2026.3.0

Root cause
OVMS requires BOTH `--tool_parser gptoss` AND `--reasoning_parser gptoss` for this model (per the official OpenVINO/gpt-oss-20b-int4-ov model card and the OVMS LLM docs). The models.json entry sets `"toolParser": "gptoss"` but does not set a reasoning parser, and the graph.pbtxt generated for the model contains no `reasoning_parser`. As a result, the "analysis" channel is never extracted.

Evidence
1) Querying the OVMS instance started by AI Playground via REST returns `content` beginning with "analysis..." and `finish_reason: "length"` (reasoning dumped into content).
2) Running OVMS manually with the two parsers added:
ovms.exe --model_name gptoss --model_path --tool_parser gptoss --reasoning_parser gptoss --target_device GPU --task text_generation --rest_port 8000
produces correct output: clean `content`, reasoning moved to `reasoning_content`, and `finish_reason: "stop"`.

Workarounds attempted (and why they fail)
- Editing the model's graph.pbtxt to add `reasoning_parser: "gptoss"`: AI Playground regenerates/overwrites graph.pbtxt on model load, so the change does not persist.
- Making graph.pbtxt read-only to prevent overwrite: OVMS then crashes on startup with exit code 1.

Steps to reproduce
1. Download the OpenVINO/gpt-oss-20b-int4-ov preset.
2. Select it, target device GPU.
3. Send any prompt in the chat.
4. Observe the error / malformed output containing the raw "analysis" channel.

Suggested fix
Pass `--reasoning_parser gptoss` (in addition to the existing tool parser) when launching OVMS for gpt-oss, i.e. include `reasoning_parser: "gptoss"` in the generated graph configuration. More generally, support a reasoning-parser field in models.json alongside `toolParser`, so reasoning models are handled correctly.

Contributor guide

Open the contributing guide

Research direction

Start with the OpenVINO/gpt-oss-20b-int4-ov entry in models.json and trace the code that generates graph.pbtxt and launches OVMS. Confirm how toolParser is passed, then verify that reasoning_parser: "gptoss" is included and that chat responses separate reasoning_content from content without malformed output.

Written by the indexing model from the issue text.

Assessment

Domain
ai
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.