thinking_budget: none is silently ignored on OpenAI-compatible endpoints, and provider_opts cannot pass the vendor off switch

Aperta
#4,309 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
52/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
go

Direzione di ricerca

Inizia da applyModelDefaults e traccia il percorso di thinking_budget e provider_opts fino ai builder delle richieste OpenAI e DMR, quindi ispeziona il percorso Model Runner per individuare cloni di MCP sampling. Verifica i body delle richieste rispetto agli scenari mlx_lm e DMR descritti, inclusi i budget disabilitati preservati e l'inoltro di extra_body; il lavoro è completo quando i controlli proposti per disabilitare il pensiero funzionano senza interrompere il comportamento esistente dei modelli di ragionamento.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

area/providers area/providers/docker-model-runner area/providers/openai

Open-weight reasoning models (Qwen3, DeepSeek, GLM) think by default, and every OpenAI-compatible server or vendor exposes a request field to turn it off: llama.cpp, vLLM, SGLang and mlx_lm accept chat_template_kwargs: {"enable_thinking": false}; llama.cpp, vLLM, SGLang, Ollama, Groq and Cerebras accept reasoning_effort: "none"; DeepSeek uses thinking: {"type": "disabled"}. docker agent can send none of them: thinking_budget is only acted on for OpenAI reasoning model names (modelinfo.UsesReasoningEffort), a disabled budget is normalised to nil in applyModelDefaults before any client sees it (which also leaves the documented DMR reasoning-budget: 0 path dead), and provider_opts forwarding is limited to the sampling allowlist. The model therefore reasons on every call and max_tokens can be spent entirely on reasoning.

Measured with v1.139.0 on macOS against mlx_lm.server 0.31.3 serving mlx-community/Qwen3.6-35B-A3B-8bit:

models:
  local:
    provider: openai
    model: mlx-community/Qwen3.6-35B-A3B-8bit
    base_url: http://localhost:8080/v1
    temperature: 0

docker agent run --exec --json agent.yaml - with a 7 kB prompt took 80–200 s per call and emitted 4.5k–13k agent_choice_reasoning events; the same prompt with chat_template_kwargs sent directly returns in about 3 s. thinking_budget: none, provider_opts: {chat_template_kwargs: {enable_thinking: false}} and a /no_think prefix all changed nothing (209–231 reasoning events on a three-word probe); a proxy confirms the body carries only model, messages, stream, temperature. With max_tokens: 2048 the whole budget went to reasoning: zero agent_choice events, exit 0, a warning event, no error. provider: dmr with runtime_flags: ["--reasoning-budget", "0"] on qwen3.6:35b-a3b-q8_0 still produced 218 reasoning events (whether the flag reached llama.cpp is unconfirmed; a failed _configure is only logged at debug level).

Proposed:

  1. provider_opts.extra_body (object) merged verbatim into the chat-completions body for the openai and dmr clients, on every provider, as the general escape hatch (extra_body: {reasoning_effort: none} on groq, extra_body: {thinking: {type: disabled}} on deepseek, ...).
  2. thinking_budget: none / 0 on a user-supplied base_url (not Azure/ChatGPT, model name not an OpenAI one) sends chat_template_kwargs: {"enable_thinking": false} on Chat Completions with the existing 256-token max_tokens floor; the dmr client sends the same per request alongside llamacpp.reasoning-budget, which also covers its MLX and SGLang engines.
  3. dmr: internal no-thinking clones (MCP sampling) no longer call the model-wide _configure. Model Runner replaces a model's stored configuration on every call, so a sampling call already wiped the agent's reasoning-budget (verified: 4096 → {} in docker model configure show); with the disabled budget preserved it would set it to 0 instead. Title and compaction clones already skipped it.

Follow-up, not in scope: a per-alias off switch in the alias registry so thinking_budget: none works on built-in aliases such as ollama (which honours reasoning_effort: none) without extra_body.

Lingua principale
Go
Stelle
3.3k
Fork
462
Merge medio
1g 10h
PR unite (30g)
273

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di docker/docker-agent

Tutte le issue di docker/docker-agent

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.