microsoft / microsoft/magentic-ui

[feature] Add OrcaRouter as a named model provider

Open
#574 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.1k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

### What problem does this solve?

`config.yaml` and the Settings UI both treat model endpoints as a generic OpenAI-compatible pair of `base_url` + `model`. That works, but for users who route through **[OrcaRouter](https://www.orcarouter.ai)** — an OpenAI-compatible model gateway with smart routing across many upstream models — it means hand-typing the endpoint every time and re-discovering the right model IDs. The repo already sets the precedent for a *named* provider (`provider: AzureOpenAIChatCompletionClient` in `src/magentic_ui/_ai_client.py`, the `model_config_azure_example` block in `config.yaml.example`, and the Azure bullet in `docs/configuration.md`). A named OrcaRouter provider would mirror that precedent so routing through the gateway is a one-block setup.

It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

### Proposed solution

Add OrcaRouter as a named model provider, mirroring the existing Azure named-provider pattern:

1. **Backend registry** — `src/magentic_ui/_ai_client.py`:
- Add an `_ORCAROUTER_PROVIDERS` set and a `_create_orcarouter()` factory alongside `_create_azure()`, dispatching on `provider: OrcaRouterChatCompletionClient`.
- The factory returns the standard `AsyncOpenAI` client with `base_url=https://api.orcarouter.ai/v1`, an `ORCAROUTER_API_KEY` env-var fallback, and a default model of `orcarouter/auto` (OrcaRouter's smart-routing model). Everything downstream (the `ChatClient` wrapper, OmniAgent, FaraWebSurfer) is already OpenAI-compatible and needs no changes.

2. **Config example** — `config.yaml.example`: add a `model_config_orcarouter` anchor block for both the orchestrator and web-surfer roles (mirroring the Azure example), so users can route either agent through the gateway.

3. **Docs** — `docs/configuration.md`: add an OrcaRouter bullet and YAML example in the "Model clients" notes (mirroring the Azure note), including that the browser-use model should be a vision-capable gateway model (e.g. `orcarouter/fusion`).

4. **Optional UI polish** — the model cards in `ModelSettings.tsx` / onboarding currently accept free-form input; a provider preset that pre-fills `https://api.orcarouter.ai/v1` + `orcarouter/auto` would make adoption one click.

I verified the gateway's OpenAI-compatible surface with a live probe:
- `GET https://api.orcarouter.ai/v1/models` → `200`, 208 models, `orcarouter/auto` present.
- `POST https://api.orcarouter.ai/v1/chat/completions` with `model: "orcarouter/auto"` → `200` with a normal `chat.completion` response (routed to `deepseek-v4-pro`), which is exactly the shape MagenticLite's `_verify_single_endpoint` probe and `create_openai_client` expect.

### Alternatives considered

- **Generic OpenAI-compatible passthrough only** (current state): already works if users type `https://api.orcarouter.ai/v1` by hand, but there's no named entry point, no preset, and no documentation — the endpoint stays invisible and each user re-discovers it. A named provider gives the same one-block config the repo already offers for Azure.
- **Frontend-only dropdown**: would be user-visible but without the backend `provider` dispatch the saved config still records a generic client; the backend named provider is the foundation, so it should come first.

### Additional context

I'm an engineer on the OrcaRouter team — happy to turn this into a PR (the code is small and mirrors the Azure wiring), and to adjust the model IDs, defaults, or docs based on your preferences. I understand from CONTRIBUTING that code PRs are currently limited to vetted contributors; opening this issue first per the repo's issues-first guidance.

Contributor guide

Open the contributing guide

Research direction

Start with the Azure provider wiring in src/magentic_ui/_ai_client.py, then compare its configuration anchor in config.yaml.example and documentation in docs/configuration.md. Check how ModelSettings.tsx and onboarding handle provider presets if taking on the optional UI work. Done means OrcaRouter dispatch, defaults, role-specific examples, and documentation are consistent, with the existing endpoint verification still passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript, yaml
Domain
backend, documentation, frontend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.