llama.cpp backend: allow `--reasoning on|off` in runtime flag allowlist
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 651
- Forks
- 155
- PR merge metrics
- No merged PRs in 30d
Description
llama.cpp's llama-server (verified on build 9879 / 72874f559) exposes a first-class server flag:
--reasoning [on|off|auto] Use reasoning/thinking in the chat (default: auto = detect from template)
docker model configure rejects it:
$ docker model configure <model> -- --reasoning off
runtime flag "--reasoning" is not allowed for backend "llama.cpp"
It's missing from pkg/inference/runtime_flags_allowlist.go (which already allows --reasoning-format, --reasoning-budget, --reasoning-preserve, --chat-template-kwargs).
Why it matters: for hybrid thinking models (Qwen 3.5/4B family), the current way to pin a model non-thinking at the server level is --chat-template-kwargs '{"enable_thinking": false}'. On current builds this logs a deprecation warning at every server start:
W Setting 'enable_thinking' via --chat-template-kwargs is deprecated. Use --reasoning on / --reasoning off instead.
The flag is a simple enum, touches no files or external resources, and fits the existing allowlist security model (cf. #1032 for the reasoning-preserve flags).
Related: #1028 (request-body reasoning_effort: none), #1032 (reasoning-preserve flags).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pkg/inference/runtime_flags_allowlist.go, where the existing llama.cpp reasoning-related flags are defined. Compare the allowlist entries with the documented llama-server --reasoning on|off|auto flag, then verify that docker model configure -- --reasoning off is accepted without the previous rejection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 90/100