thinking_budget: none is silently ignored on OpenAI-compatible endpoints, and provider_opts cannot pass the vendor off switch
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 3.3k
- Forks
- 462
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 273
Description
Open-weight reasoning models (Qwen3, DeepSeek, GLM) think by default, and every OpenAI-compatible server or vendor exposes a request field to turn it off: llama.cpp, vLLM, SGLang and mlx_lm accept chat_template_kwargs: {"enable_thinking": false}; llama.cpp, vLLM, SGLang, Ollama, Groq and Cerebras accept reasoning_effort: "none"; DeepSeek uses thinking: {"type": "disabled"}. docker agent can send none of them: thinking_budget is only acted on for OpenAI reasoning model names (modelinfo.UsesReasoningEffort), a disabled budget is normalised to nil in applyModelDefaults before any client sees it (which also leaves the documented DMR reasoning-budget: 0 path dead), and provider_opts forwarding is limited to the sampling allowlist. The model therefore reasons on every call and max_tokens can be spent entirely on reasoning.
Measured with v1.139.0 on macOS against mlx_lm.server 0.31.3 serving mlx-community/Qwen3.6-35B-A3B-8bit:
models:
local:
provider: openai
model: mlx-community/Qwen3.6-35B-A3B-8bit
base_url: http://localhost:8080/v1
temperature: 0
docker agent run --exec --json agent.yaml - with a 7 kB prompt took 80–200 s per call and emitted 4.5k–13k agent_choice_reasoning events; the same prompt with chat_template_kwargs sent directly returns in about 3 s. thinking_budget: none, provider_opts: {chat_template_kwargs: {enable_thinking: false}} and a /no_think prefix all changed nothing (209–231 reasoning events on a three-word probe); a proxy confirms the body carries only model, messages, stream, temperature. With max_tokens: 2048 the whole budget went to reasoning: zero agent_choice events, exit 0, a warning event, no error. provider: dmr with runtime_flags: ["--reasoning-budget", "0"] on qwen3.6:35b-a3b-q8_0 still produced 218 reasoning events (whether the flag reached llama.cpp is unconfirmed; a failed _configure is only logged at debug level).
Proposed:
provider_opts.extra_body(object) merged verbatim into the chat-completions body for the openai and dmr clients, on every provider, as the general escape hatch (extra_body: {reasoning_effort: none}on groq,extra_body: {thinking: {type: disabled}}on deepseek, ...).thinking_budget: none/0on a user-suppliedbase_url(not Azure/ChatGPT, model name not an OpenAI one) sendschat_template_kwargs: {"enable_thinking": false}on Chat Completions with the existing 256-tokenmax_tokensfloor; the dmr client sends the same per request alongsidellamacpp.reasoning-budget, which also covers its MLX and SGLang engines.- dmr: internal no-thinking clones (MCP sampling) no longer call the model-wide
_configure. Model Runner replaces a model's stored configuration on every call, so a sampling call already wiped the agent'sreasoning-budget(verified: 4096 →{}indocker model configure show); with the disabled budget preserved it would set it to 0 instead. Title and compaction clones already skipped it.
Follow-up, not in scope: a per-alias off switch in the alias registry so thinking_budget: none works on built-in aliases such as ollama (which honours reasoning_effort: none) without extra_body.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at applyModelDefaults and trace how thinking_budget and provider_opts reach the OpenAI and DMR request builders, then inspect the Model Runner path for MCP sampling clones. Verify the request bodies against the mlx_lm and DMR scenarios described, including preserved disabled budgets and extra_body forwarding; done means the proposed no-thinking controls work without breaking existing reasoning-model behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- ai, api, backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100