microsoft / microsoft/foundry-local

complete_chat raises an opaque pydantic ValidationError when the model smollm3-3b returns a malformed tool call

Open
#864 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.6k
Forks
369
Avg merge
2d 17h
Merged PRs (30d)
39

Description

**Summary**

When a loaded model emits a malformed `tool_call` in its response (e.g. a tool call
with a missing `function.name` and `arguments` set to the string `"null"`),
`ChatClient.complete_chat()` crashes with a raw pydantic `ValidationError` instead of a
clear, catchable SDK error. The same unguarded `model_validate` call exists in the
streaming path. A single misbehaving model therefore aborts the whole application with an
error that gives no hint about the actual cause (the model's output).

**Environment**

- `foundry-local-sdk` 1.2.1 (also present on `main`)
- Python 3.13, macOS
- Reproduced with model `smollm3-3b` while passing `tools=[...]`

**What happens**

`smollm3-3b`, during a tool-calling conversation, returned a second tool call with no
function name and `arguments == "null"`:

```
choices.0.message.tool_calls.1.function.name
Field required [type=missing, input_value={'arguments': 'null'}, input_type=dict]
```

which surfaces as:

```
pydantic_core._pydantic_core.ValidationError: 3 validation errors for ChatCompletion
choices.0.message.tool_calls.1.ChatCompletionMessageFunctionToolCall.function.name
Field required [type=missing, input_value={'arguments': 'null'}, input_type=dict]
choices.0.message.tool_calls.1.ChatCompletionMessageCustomToolCall.custom
Field required [type=missing, ...]
choices.0.message.tool_calls.1.ChatCompletionMessageCustomToolCall.type
Input should be 'custom' [type=literal_error, input_value='function', ...]
```

Stack (abridged) — the crash is at `chat_client.py` `complete_chat` →
`ChatCompletion.model_validate_json(response_json)`:

```
File ".../foundry_local_sdk/openai/chat_client.py", line 219, in complete_chat
completion = ChatCompletion.model_validate_json(response.data)
File ".../pydantic/main.py", line 782, in model_validate_json
return cls.__pydantic_validator__.validate_json(...)
pydantic_core._pydantic_core.ValidationError: 3 validation errors for ChatCompletion
```

**Minimal, model-free repro** (shows the SDK has no guard around response validation):

```python
from openai.types.chat import ChatCompletion

# A response containing a malformed second tool call (missing function.name,
# arguments == "null") — exactly what smollm3-3b emitted through Foundry Local.
malformed = '''{
"id": "chatcmpl-x", "object": "chat.completion", "created": 0, "model": "smollm3-3b",
"choices": [{
"index": 0, "finish_reason": "tool_calls",
"message": {
"role": "assistant", "content": null,
"tool_calls": [
{"index": 0, "id": "a", "type": "function",
"function": {"name": "get_weather", "arguments": "{}"}},
{"index": 1, "id": "b", "type": "function",
"function": {"arguments": "null"}}
]
}
}]
}'''

ChatCompletion.model_validate_json(malformed) # -> raw pydantic ValidationError
```

**Expected behavior**

The docstring already promises `FoundryLocalException` on failure. The SDK should either:

1. **(minimum)** wrap the parse and re-raise a clear `FoundryLocalException` that includes
the raw response payload, so the error is catchable and actionable; and/or
2. **(nicer)** tolerate a malformed tool call — drop/skip the invalid entry and log a
warning — so one bad tool call doesn't abort an otherwise-usable response.

**Why it matters**

Smaller models are exactly the ones Foundry Local targets, and they routinely emit
imperfect tool calls. The SDK crashing with an internal pydantic traceback (rather than a
clear SDK error) makes tool-calling apps brittle and hard to debug.

Contributor guide

Open the contributing guide

Research direction

Start in chat_client.py at ChatClient.complete_chat and the corresponding streaming validation path, where ChatCompletion.model_validate_json(response.data) is called. Review the docstring's FoundryLocalException contract and reproduce the malformed tool-call payload; done means malformed responses no longer expose a raw pydantic ValidationError and the behavior is covered for both paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.