microsoft / microsoft/agent-framework
Python: [Feature]: make tool argument-validation errors self-correcting for the model
- Dominant language
- Python
- Stars
- 13.6k
- Forks
- 2.3k
- Avg merge
- 2d 45m
- Merged PRs (30d)
- 358
Description
### Description
## Problem
When a model calls a tool with arguments that fail schema validation, the function result sent back to the model is, by default, exactly:
```
Error: Argument parsing failed.
```
This names neither the tool's expected parameters, nor the offending/missing keys, nor even the fact that the *shape* (vs. the values) was wrong. A model that made a systematic shape error has nothing to correct against, so it plausibly retries the identical call.
**Live evidence (deployed AG-UI app, Mistral Medium 3.5, 2026-07-16, thread `b48ceff3`):** the model called the harness todo tool as
```json
{"items": [{"id": 4, "title": "Create architecture recommendation plan document ..."}]}
```
where `todos_add` expects `{"todos": [{"title": ..., "description": ...}]}` — it mimicked the shared-state snapshot's `todo.items[]` representation instead of the declared tool schema. It received `Error: Argument parsing failed.` and retried the **byte-identical** call four times (23:01:12, 23:01:29, 23:08:05, 23:09:01), failing every time. With reasoning enabled each roundtrip cost 1–2.5 minutes, so the run degenerated into a slow livelock: plan complete, todo list never finalized, turn never ending — bounded only by `max_function_calls`. The model never deviated because the error gave it nothing to deviate *toward*.
## Why the existing knob is not enough
`function_invocation_configuration["include_detailed_errors"] = True` (default `False`) appends `Exception: {exc}` — the raw `pydantic.ValidationError`/`TypeError` text. That is usually sufficient for self-correction (pydantic names missing/extra fields), and it is what we are enabling app-side as a workaround. But:
1. **The default is the trap.** The flag is easy to miss, and the safe-looking default silently converts a one-shot model mistake into an N × roundtrip livelock. Most integrators will discover the flag only after diagnosing one.
2. **Raw exception text is developer-oriented**, not model-oriented: multi-line pydantic dumps with `type=extra_forbidden`, `input_value=...`, URLs to pydantic docs — tokens spent on noise, and `input_value` echoes payload content back verbatim.
3. **The leak rationale doesn't apply to this branch.** Guarding exception detail behind a flag makes sense for tool *execution* errors (the handlers at ~1531/1607 can leak application internals). Argument-*validation* errors are different: the failing data is the model's own output, and the schema is already in the model's context as the tool declaration. There is nothing in "unexpected key `items`; expected key `todos`" that the model does not already have.
## Requested
Make the **default** argument-validation error actionable, e.g. any of:
1. A structured, model-oriented summary built from the validation error and the tool's schema — no raw exception repr, no input echo:
```
Error: invalid arguments for tool 'todos_add': unexpected key 'items'; missing
required key 'todos'. Expected parameters: {"todos": [{"title": str, "description":
str (optional)}]}.
```
2. Or, minimally: include the tool name + the top-level offending/missing key names in the default message, keeping full exception text behind `include_detailed_errors`.
3. Optionally: a `arg_error_detail: Literal["generic", "keys", "schema", "exception"]` knob replacing the boolean, defaulting to `"keys"`.
Related hardening (separate knob, possibly separate issue): a repeated-identical-failure guard — same tool + same arguments + same validation error N times within a run → escalate the error content (append the full schema) or refuse the call, so a non-correcting model cannot burn `max_function_calls` × provider-latency on one mistake.
## Workaround (application-side, today)
Set `function_invocation_configuration={"include_detailed_errors": True}` on the chat client (our `ats.providers._function_invocation_configuration` already owns this dict). Trade-off accepted: raw exception text in tool results, including for tool-execution errors where it may expose internals — acceptable for this app, wrong as a general default, which is why the ask above is about upstream defaults rather than the flag.
### Code Sample
```markdown
```
### Language/SDK
Python
Contributor guide
Assessment
This issue has not been assessed yet.