ollama / ollama/ollama-python

Add tool_choice parameter to chat() (mirror server-side ollama#11171)

Open
#663 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
10.5k
Forks
1.2k
Avg merge
4m
Merged PRs (30d)
1

Description

Observation

The chat() function and underlying ChatRequest model in ollama-python do not expose a tool_choice parameter, even though several upstream endpoints already handle it:

  • OpenAI-compat endpoint in ollama/ollama server (openai/responses.go): ToolChoice is parsed off the request, with TODO comments noting partial support.
  • Anthropic-compat endpoint (anthropic/trace.go + anthropic/anthropic.go): ToolChoice is wired through.
  • Native /api/chat: tool_choice is not a request field today; tracked upstream at ollama/ollama#11171 ("Add support for tool_choice set to any to require a model to use a tool").
Empirical behavior today
import ollama

# Test 1: tool_choice as kwarg → hard rejection
ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Hello"}],
    tool_choice="auto",
)
# TypeError: Client.chat() got an unexpected keyword argument 'tool_choice'

# Test 2: tool_choice via options dict → silently accepted but not wired
# (options is passed through as model parameters; tool_choice has no effect there)
ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Hello"}],
    options={"tool_choice": "auto"},
)
# OK, no error, but tool_choice is not interpreted by the server

ChatRequest.model_json_schema() confirms there is no tool_choice field. Current fields: model, stream, options, format, keep_alive, messages, tools, think, logprobs, top_logprobs.

Why it matters

Practitioners porting code from OpenAI's or Anthropic's SDK to ollama-python expect a tool_choice parameter (to force tool use, restrict to a specific tool, or disable tools). Today the only available control is omitting tools entirely, which is coarse.

Concrete suggestion

When the native /api/chat server endpoint adds tool_choice (per ollama/ollama#11171), expose the same parameter on ollama.chat() and ChatRequest:

# Standard cross-vendor values
tool_choice: Literal["auto", "required", "none"] | dict | None = None

Until then, this issue serves as a tracking item for the Python client mirror so it lands in the same release window as the server-side feature.

Workarounds
  • For "must use a tool" semantics, append a directive to the system prompt and use temperature=0.0 (verified to produce reliable tool calls on Llama 3.1 8B).
  • For "must not use a tool", omit the tools argument entirely.

Happy to send a PR once the server-side direction is settled.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the chat() entry point and the ChatRequest model, then inspect ChatRequest.model_json_schema() to confirm the current request surface. Track ollama/ollama#11171 and the mentioned OpenAI-compat and Anthropic-compat paths before proceeding; done means the Python client mirrors the settled native server behavior for tool_choice.

Written by the indexing model from the issue text.

Assessment

Tech stack
ollama, python
Domain
api
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.