[Umbrella] Pluggable model routing (MODEL_ROUTER): delivery status, Python parity, documentation, and follow-ups
- Dominant language
- Java
- Stars
- 452
- Forks
- 167
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 49
Description
Umbrella issue to track pluggable in-chat model routing end to end, as requested in #1042.
**Design:** discussion #897 — select-not-delegate, the framework owns the model call; judge-style policies run on the engine's durable, metered chat path.
**Delivered in v1 (#964, merged):**
- `RoutingStrategy` SPI and the `MODEL_ROUTER` resource; `Strategies.rules(...)` and `Strategies.of(...)`
- Durable decision persistence and replay (ids `route:` / `chat::`), route-once-per-reasoning-loop
- Fallback through declared candidate order; `ModelRoutingEvent` (observability-only) with decision metadata and latency histogram
- Build-time validation (regex compile, null/template checks)
**v2 (#1042, in review):**
- Framework-managed LLM-as-judge: `Strategies.llm(judgeModel[, promptTemplate])`; the engine executes the judge call durably (`judge:`) with token attribution and events
- Verdicts constrained to candidate names; unparseable/non-candidate replies abstain to the default model; judge failures honor the error-handling strategy; `decision_source` recorded
- Plan-time validation of judge references; `ModelRoutingJudgeExample`
**Follow-ups to track:**
- [ ] User documentation for model routing (v1 + v2) — covers the `doc-needed` label
- [ ] Python-side parity (today `add_resource(..., MODEL_ROUTER)` raises with a clear error; cross-language plan deserialization is already tested)
Contributor guide
Research direction
Start with design discussion #897, delivered work in #964, and the v2 review in #1042. Determine the documentation entry points for v1 and v2 and the Python path where add_resource(..., MODEL_ROUTER) currently raises; review the existing cross-language plan deserialization test. Done means user documentation exists and Python-side model-router parity is implemented and tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- ai, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100