Add tool_choice parameter to chat() (mirror server-side ollama#11171)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 10.5k
- Forks
- 1.2k
- Avg merge
- 4m
- Merged PRs (30d)
- 1
Description
Observation
The chat() function and underlying ChatRequest model in ollama-python do not expose a tool_choice parameter, even though several upstream endpoints already handle it:
- OpenAI-compat endpoint in
ollama/ollamaserver (openai/responses.go):ToolChoiceis parsed off the request, with TODO comments noting partial support. - Anthropic-compat endpoint (
anthropic/trace.go+anthropic/anthropic.go):ToolChoiceis wired through. - Native
/api/chat:tool_choiceis not a request field today; tracked upstream at ollama/ollama#11171 ("Add support fortool_choiceset toanyto require a model to use a tool").
Empirical behavior today
import ollama
# Test 1: tool_choice as kwarg → hard rejection
ollama.chat(
model="llama3.1:8b",
messages=[{"role": "user", "content": "Hello"}],
tool_choice="auto",
)
# TypeError: Client.chat() got an unexpected keyword argument 'tool_choice'
# Test 2: tool_choice via options dict → silently accepted but not wired
# (options is passed through as model parameters; tool_choice has no effect there)
ollama.chat(
model="llama3.1:8b",
messages=[{"role": "user", "content": "Hello"}],
options={"tool_choice": "auto"},
)
# OK, no error, but tool_choice is not interpreted by the server
ChatRequest.model_json_schema() confirms there is no tool_choice field. Current fields: model, stream, options, format, keep_alive, messages, tools, think, logprobs, top_logprobs.
Why it matters
Practitioners porting code from OpenAI's or Anthropic's SDK to ollama-python expect a tool_choice parameter (to force tool use, restrict to a specific tool, or disable tools). Today the only available control is omitting tools entirely, which is coarse.
Concrete suggestion
When the native /api/chat server endpoint adds tool_choice (per ollama/ollama#11171), expose the same parameter on ollama.chat() and ChatRequest:
# Standard cross-vendor values
tool_choice: Literal["auto", "required", "none"] | dict | None = None
Until then, this issue serves as a tracking item for the Python client mirror so it lands in the same release window as the server-side feature.
Workarounds
- For "must use a tool" semantics, append a directive to the system prompt and use
temperature=0.0(verified to produce reliable tool calls on Llama 3.1 8B). - For "must not use a tool", omit the
toolsargument entirely.
Happy to send a PR once the server-side direction is settled.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the chat() entry point and the ChatRequest model, then inspect ChatRequest.model_json_schema() to confirm the current request surface. Track ollama/ollama#11171 and the mentioned OpenAI-compat and Anthropic-compat paths before proceeding; done means the Python client mirrors the settled native server behavior for tool_choice.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ollama, python
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100