RFC: Unified model-routing layer — one config that covers per-role, per-tool, per-turn, and capability routing

Summary

There are currently five open requests asking jcode to route different kinds of work to different models, each describing a separate axis:

Every one of these builds the same engine (pick a different model for a request based on some condition) with a different config surface. The maintainer would have to design and test this five times. #861 already being marked duplicate shows the overlap is visible.

This RFC proposes one routing layer with orthogonal, composable policies, so all five requests (and future ones) fall out of a single config table and a single evaluation point. It also composes with the sister proposals (#471, #771, #566, #96, #833) rather than competing with them.

Current state

• jcode already knows per-model capability (provider.supports_image_input()) and has a complete_with_failover request chokepoint where a routing decision can live (#861 cites it; #755 is the marker-replacement behavior to preserve as the no-fallback safety net).
• Routing today is static and manual: one active model, Ctrl+Tab to switch. Swarm spawns already accept a per-worker model but there is no role-level config (#683).
• #833 documents a hard constraint any dynamic-injection feature must respect: mutable content injected into the cached system prefix breaks the KV cache (~97K tokens re-sent per turn). The routing layer must therefore decide statically per turn, not by mutating shared prompt state.

Proposed design

A single [routing] section with orthogonal policies, evaluated in a defined order at the existing request chokepoint:

[routing]
# 1. Capability routing (#819, #861)
#    Turn contains image content and active model is text-only?
#    -> route that turn to vision_model, then switch back.
#    No vision_model configured -> keep current behavior (#755 marker). Never leak image data.
vision = { on_image_input = "provider/vision-model", fallback = "provider/text-only-model" }

# 2. Effort/task routing (#121)
#    Turns with no user prompt and only read/search/tool work -> routine_model.
routine = { model = "provider/fast-model", match = "tool-only" }

# 3. Role routing for swarm (#683)
[swarm.roles.coder]
model = "provider/fast-model"
[swarm.roles.reviewer]
model = "provider/strong-model"

And a per-tool override table (#810):

[[routing.tool_override]]
tools = ["agentgrep", "websearch", "webfetch", "browser"]
provider = "zai-coding-plan"
model = "GLM-4.7"

Precedence (must be explicit in the RFC and in tests)

1. Capability routing (safety: never send image data to a text-only model).
2. Per-tool override (narrowest scope wins for that one call).
3. Role routing (applies at swarm spawn time).
4. Effort routing (applies to tool-only turns).
5. Active model (default).

Evaluation point

All policies are evaluated once, per turn, at the existing complete_with_failover chokepoint. No shared prompt prefix is mutated, so the KV cache is untouched (#833 constraint). The decision is recorded on the turn and visible to the user ("handled by X" notice, as #861 asks).

How this composes with the sister proposals (not a mega-feature)

• #471 (LSP / semantic feedback): the "routine vs thinking" classifier gets real signal. A turn that just triggered diagnostics-on-edit is not routine, even if it is tool-only. The read-before-edit guard from #471 is the natural safety companion to routing cheap models at routine work.
• #771 / #566 / #96 (permissions): routing must inherit the permission layer, not bypass it. Per-tool routing runs the same allowlist/approval gates; capability routing's vision_model is a named, configured route, never an arbitrary model-supplied one.
• #833 (host-level shared state): a cheap model is the right engine for digesting other sessions' state. The routing layer gives #833 an official place to put that model.
• #144 / #686 (client/server, continuation): routing is a server-side decision; once it lives in jcode-app-core, headless runs and swarm inherit it for free, matching #686's ask to move continuation server-side.

Acceptance criteria

• One [routing] config surface; no new per-axis config formats for the five existing requests.
• All five existing requests (#121, #683, #810, #819, #861) are satisfiable by combinations of the knobs above, with no new code per request.
• Capability routing never sends image data to a text-only model when no vision model is configured (current #755 behavior preserved).
• Routing decision is static per turn; the cached system prefix is never mutated mid-session (#833 constraint).
• The user is notified when a turn was handled by a routed (non-active) model.
• Precedence, fallback, and no-fallback cases are covered by tests at the complete_with_failover chokepoint.
• Per-tool routing inherits the permission layer from #771/#566 (same gates, not a bypass).

Related

• Routing cluster: #121, #683, #810, #819, #861, #755
• Composes with: #471, #771, #566, #96, #833, #144, #686

Start with the existing complete_with_failover request chokepoint and the jcode-app-core path mentioned in the RFC. Trace current model selection, capability checks, failover, permission gates, and user notices before defining the unified routing surface. Done means the five routing cases, precedence, fallback behavior, cache constraint, notices, and chokepoint tests are covered.