docker / docker/model-runner

llama.cpp backend: allow `--reasoning on|off` in runtime flag allowlist

Open Beginner friendly
#1,064 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
651
Forks
155
PR merge metrics
No merged PRs in 30d

Description

llama.cpp's llama-server (verified on build 9879 / 72874f559) exposes a first-class server flag:

--reasoning [on|off|auto]   Use reasoning/thinking in the chat (default: auto = detect from template)

docker model configure rejects it:

$ docker model configure <model> -- --reasoning off
runtime flag "--reasoning" is not allowed for backend "llama.cpp"

It's missing from pkg/inference/runtime_flags_allowlist.go (which already allows --reasoning-format, --reasoning-budget, --reasoning-preserve, --chat-template-kwargs).

Why it matters: for hybrid thinking models (Qwen 3.5/4B family), the current way to pin a model non-thinking at the server level is --chat-template-kwargs '{"enable_thinking": false}'. On current builds this logs a deprecation warning at every server start:

W Setting 'enable_thinking' via --chat-template-kwargs is deprecated. Use --reasoning on / --reasoning off instead.

The flag is a simple enum, touches no files or external resources, and fits the existing allowlist security model (cf. #1032 for the reasoning-preserve flags).

Related: #1028 (request-body reasoning_effort: none), #1032 (reasoning-preserve flags).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in pkg/inference/runtime_flags_allowlist.go, where the existing llama.cpp reasoning-related flags are defined. Compare the allowlist entries with the documented llama-server --reasoning on|off|auto flag, then verify that docker model configure -- --reasoning off is accepted without the previous rejection.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend
Issue type
Feature
Difficulty
1/5
Estimated time
Under an hour
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
90/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.