microsoft / microsoft/magentic-ui
[bug] OmniAgent drops MagenticBrain tool calls when the server peels them into message.tool_calls
- Dominant language
- Python
- Stars
- 10.1k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
### Describe the bug
OmniAgent only parses MagenticBrain tool use from `message.content` (`{...}` XML). Some OpenAI-compatible servers detect those markers, move them into native `message.tool_calls`, and leave only the model's prose in `content`.
OmniAgent then treats the turn as bare text: one nudge, then a "final answer" that narrates using the tool instead of calling it. Sub-agents (e.g. Fara via `delegate_cua`) never start.
This is not MagenticBrain failing to emit tool calls — the server already has them in `tool_calls`. The client just never reads that field.
The documented vLLM / Hugging Face / Foundry path is likely unaffected (those servers typically leave the XML in `content`). The bug shows up with servers that implement native tool-call extraction, including `mlx_lm.server`.
### Steps to reproduce
1. Serve MagenticBrain through an OpenAI-compatible endpoint that extracts `` into `message.tool_calls` (reproduced with `mlx_lm.server --model mlx-community/MagenticBrain-8bit`).
2. Point MagenticLite's orchestrator `base_url` at that `/v1` endpoint.
3. Ask a task that should dispatch the web agent, e.g. "Go to https://example.com and describe the homepage."
4. Optionally confirm the server *did* emit native tool calls:
```python
from openai import OpenAI
c = OpenAI(base_url="http://127.0.0.1:8100/v1", api_key="not-needed")
r = c.chat.completions.create(
model="mlx-community/MagenticBrain-8bit",
messages=[
{"role": "system", "content": "...MagenticLite tool prompt with delegate_cua..."},
{"role": "user", "content": "Go to https://example.com and describe the homepage."},
],
)
print(r.choices[0].message.content)
print(r.choices[0].message.tool_calls)
print(r.choices[0].finish_reason) # often "tool_calls"
### Expected behavior
OmniAgent parses a delegate_cua (or other) tool call and runs it. The UI shows Using tool: delegate_cua, and Fara starts.
### Actual behavior
### **UI loop:**
• **Reasoning**: "This is a web browsing task, so I should use the delegate_cua function..."
• **Nudge**: response did not contain or
• **Final answer**: "I need to delegate the web browsing task to the web agent. Let me use the delegate_cua function with the appropriate parameters."
Fara never runs. Trace shows content is the prose only; finish_reason on the raw completion is tool_calls.
### MagenticLite version
0.2.2
### Operating system
macOS
### Python version
3.12
### Model client / provider
`mlx-community/MagenticBrain-8bit` via `mlx_lm.server` & `mlx-community/Fara1.5-9B-8bit` via `mlx_vlm.server`
### Additional context
**Root cause**: `OmniResponses._call_api` takes `response.choices[0].message.content or ""` and `parse_response()` only looks for `` in that string. Native `message.tool_calls` is ignored.
**Suggested fix**: If `tool_calls` is present and `content` does not already contain `` tags, reconstruct the XML MagenticLite already understands, then parse as today. If content already has the tags (vLLM path), leave it alone.
I prototyped this in `zboyles/magentic-ui` branch `fix/rehydrate-native-tool-calls` (commit [ece97a2](https://github.com/zboyles/magentic-ui/commit/ece97a2)): `rehydrate_native_tool_calls()` in `_parse.py`, used from `_responses.py`, plus tests in `tests/agents/omni/test_parse.py`. Ran several live checks successfully, the same browsing task emitted `delegate_cua` and started Fara.
I can adapt that patch if you want to take it in-tree; I created this issue first per `CONTRIBUTING.md`.
Contributor guide
Research direction
Start with OmniResponses._call_api in _responses.py and the response parsing in _parse.py; compare how message.content and native message.tool_calls are handled. Read the existing tests in tests/agents/omni/test_parse.py and run them first. Done means native tool calls are parsed without breaking content that already contains XML tool-call tags, with tests covering both paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend-api-design
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100