microsoft / microsoft/foundry-local
tool_choice="auto" returns tool calls as text with empty tool_calls (only "required" yields structured tool_calls)
- Dominant language
- C++
- Stars
- 2.6k
- Forks
- 369
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 39
Description
### Summary
When calling the OpenAI-compatible `/v1/chat/completions` endpoint with `tool_choice="auto"` (the OpenAI default, and what most agent frameworks send), Foundry Local returns the tool call **as raw text inside `message.content`** with an **empty `tool_calls` array** and `finish_reason="stop"`. The structured `tool_calls` are only populated when the caller explicitly sets `tool_choice="required"`.
This breaks standard OpenAI clients and agent frameworks (e.g. anything built on the OpenAI SDK, LangChain, custom agents), because they read `message.tool_calls` and — per the OpenAI spec — expect `tool_choice="auto"` to let the model decide *and still return structured tool calls* when it chooses to call a tool. With Foundry Local they instead receive prose containing an XML/JSON blob that they cannot dispatch.
### Reproduction
Model: `qwen2.5-coder-7b-instruct-generic-cpu` (also reproduces on other tool-capable models), Foundry Local `0.10.0`.
Request (only `tool_choice` differs):
```bash
curl -s http://127.0.0.1:/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "qwen2.5-coder-7b-instruct-generic-cpu",
"messages": [{"role": "user", "content": "What is the weather in Paris? Use the tool."}],
"tools": [{"type": "function", "function": {
"name": "get_weather",
"description": "Get weather for a city",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}],
"tool_choice": "auto",
"temperature": 0
}'
```
### Actual behavior
**`tool_choice="auto"`** — tool call leaks into content as text, `tool_calls` empty:
```
finish_reason: stop
tool_calls: []
content: "```xml\n\n {\"name\": \"get_weather\", \"arguments\": {\"city\": \"Paris\"}}\n\n```"
```
**`tool_choice="required"`** — structured tool call is returned as expected:
```
finish_reason: tool_calls
tool_calls: [{"index": 0, "id": "09xikmhSQ", "type": "function",
"function": {"name": "get_weather", "arguments": "{\n \"city\": \"Paris\"\n}"}}]
```
### Expected behavior
With `tool_choice="auto"`, when the model decides to call a tool, Foundry Local should parse the model's tool-call output and populate `message.tool_calls` with `finish_reason="tool_calls"` — the same as `tool_choice="required"` — instead of returning the call as raw text. `auto` should differ from `required` only in *whether* a tool must be called, not in *how* a chosen call is surfaced.
### Impact
- Any OpenAI-compatible agent/framework that sends the default `tool_choice="auto"` cannot use Foundry Local tool calling without a custom text-scraping workaround.
- Forcing `tool_choice="required"` is not a general fix, because it prevents the model from answering directly when no tool is needed.
### Environment
- Foundry Local: `0.10.0`
- Endpoint: OpenAI-compatible `/v1/chat/completions`
- Models: `qwen2.5-coder-7b-instruct-generic-cpu` (reproduces broadly across tool-capable models)
- OS: Linux
### Question
Is the `auto` → text vs `required` → structured behavior intentional, or a gap in the server-side tool-call parsing? Ideally `auto` would run the same tool-call extraction that `required` already does and emit structured `tool_calls`.
Contributor guide
Research direction
Start by reproducing the issue with the documented /v1/chat/completions curl request, comparing tool_choice="auto" with "required". Trace the server-side tool-call parsing and response construction; done means auto produces structured message.tool_calls and finish_reason="tool_calls" when the model chooses a tool, while still allowing a direct response when no tool is needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai, api, backend-api-design
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100