microsoft / microsoft/foundry-local
complete_chat raises an opaque pydantic ValidationError when the model smollm3-3b returns a malformed tool call
- Dominant language
- C++
- Stars
- 2.6k
- Forks
- 369
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 39
Description
**Summary**
When a loaded model emits a malformed `tool_call` in its response (e.g. a tool call
with a missing `function.name` and `arguments` set to the string `"null"`),
`ChatClient.complete_chat()` crashes with a raw pydantic `ValidationError` instead of a
clear, catchable SDK error. The same unguarded `model_validate` call exists in the
streaming path. A single misbehaving model therefore aborts the whole application with an
error that gives no hint about the actual cause (the model's output).
**Environment**
- `foundry-local-sdk` 1.2.1 (also present on `main`)
- Python 3.13, macOS
- Reproduced with model `smollm3-3b` while passing `tools=[...]`
**What happens**
`smollm3-3b`, during a tool-calling conversation, returned a second tool call with no
function name and `arguments == "null"`:
```
choices.0.message.tool_calls.1.function.name
Field required [type=missing, input_value={'arguments': 'null'}, input_type=dict]
```
which surfaces as:
```
pydantic_core._pydantic_core.ValidationError: 3 validation errors for ChatCompletion
choices.0.message.tool_calls.1.ChatCompletionMessageFunctionToolCall.function.name
Field required [type=missing, input_value={'arguments': 'null'}, input_type=dict]
choices.0.message.tool_calls.1.ChatCompletionMessageCustomToolCall.custom
Field required [type=missing, ...]
choices.0.message.tool_calls.1.ChatCompletionMessageCustomToolCall.type
Input should be 'custom' [type=literal_error, input_value='function', ...]
```
Stack (abridged) — the crash is at `chat_client.py` `complete_chat` →
`ChatCompletion.model_validate_json(response_json)`:
```
File ".../foundry_local_sdk/openai/chat_client.py", line 219, in complete_chat
completion = ChatCompletion.model_validate_json(response.data)
File ".../pydantic/main.py", line 782, in model_validate_json
return cls.__pydantic_validator__.validate_json(...)
pydantic_core._pydantic_core.ValidationError: 3 validation errors for ChatCompletion
```
**Minimal, model-free repro** (shows the SDK has no guard around response validation):
```python
from openai.types.chat import ChatCompletion
# A response containing a malformed second tool call (missing function.name,
# arguments == "null") — exactly what smollm3-3b emitted through Foundry Local.
malformed = '''{
"id": "chatcmpl-x", "object": "chat.completion", "created": 0, "model": "smollm3-3b",
"choices": [{
"index": 0, "finish_reason": "tool_calls",
"message": {
"role": "assistant", "content": null,
"tool_calls": [
{"index": 0, "id": "a", "type": "function",
"function": {"name": "get_weather", "arguments": "{}"}},
{"index": 1, "id": "b", "type": "function",
"function": {"arguments": "null"}}
]
}
}]
}'''
ChatCompletion.model_validate_json(malformed) # -> raw pydantic ValidationError
```
**Expected behavior**
The docstring already promises `FoundryLocalException` on failure. The SDK should either:
1. **(minimum)** wrap the parse and re-raise a clear `FoundryLocalException` that includes
the raw response payload, so the error is catchable and actionable; and/or
2. **(nicer)** tolerate a malformed tool call — drop/skip the invalid entry and log a
warning — so one bad tool call doesn't abort an otherwise-usable response.
**Why it matters**
Smaller models are exactly the ones Foundry Local targets, and they routinely emit
imperfect tool calls. The SDK crashing with an internal pydantic traceback (rather than a
clear SDK error) makes tool-calling apps brittle and hard to debug.
Contributor guide
Research direction
Start in chat_client.py at ChatClient.complete_chat and the corresponding streaming validation path, where ChatCompletion.model_validate_json(response.data) is called. Review the docstring's FoundryLocalException contract and reproduce the malformed tool-call payload; done means malformed responses no longer expose a raw pydantic ValidationError and the behavior is covered for both paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100