Hmbown / Hmbown/Codewhale

feat(auto): sticky route binding — plan the auto route once per session, not once per turn

Open
#6,252 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
41k
Forks
3.6k
Avg merge
13h 59m
Merged PRs (30d)
299

Description

## Problem

The auto model router re-resolves on **every user message**: `plan_turn_route` (`crates/tui/src/turn_route_plan.rs:143`) runs the classifier (`:146-166`) per turn, and `[auto] cross_provider` (#4411) allows hopping providers turn-to-turn.

Every provider/model change forces a declared prefix-cache re-pin (`crates/tui/src/core/engine.rs:7341-7374`) — a guaranteed full cache miss on the next turn. With cache reads priced far below fresh input on every major provider, a per-turn auto route that switches models can cost more than the cheap model saves. Auto routing must be **session-sticky**: route once, stay bound.

## Change

1. `crates/tui/src/turn_route_plan.rs:143` — consult the session binding first. If bound and the binding still revalidates against `ModelInventory` (`crates/tui/src/model_inventory.rs:77` — credential present, provider runnable, readiness), skip the classifier entirely and resolve the bound provider+model.
2. `crates/tui/src/session_manager.rs:935` — promote `SavedSession.last_auto_route` (today: write-only provenance, set at `tui/ui/event_loop.rs:2584` / `dispatch.rs:933`) to a load-bearing binding read at plan time. Already `Option` with serde defaults, so migration-safe.
3. Rebind only on: binding fails revalidation, the bound route errors at dispatch (`client.rs:1421` / `route/resolver.rs:218` re-resolution), or the user explicitly changes model/provider (already a declared re-pin).
4. `[auto] cross_provider` becomes a **bind-time scope**, not a per-turn permission: it widens the candidate set when the binding is created, not afterwards.

## Already true, keep it that way

Transport retries and stream-resume never re-plan or switch provider (`client.rs:3570`, `turn_loop.rs:1786-1941`) — failure handling stays on the bound route.

## Acceptance

- A session on `model = "auto"` makes at most one classifier call per binding; subsequent turns show `routing_source: StickyBinding` (#6253).
- No prefix-cache re-pin occurs between turns that stay on the binding (verify via `PrefixStabilityManager` drift attribution, `crates/core/src/prefix_cache.rs`).
- Removing the bound provider credential and resuming the session triggers exactly one rebind, receipted.
- Explicit user model change busts the binding deliberately and is recorded as such.

Contributor guide

Open the contributing guide

Research direction

Start with plan_turn_route in crates/tui/src/turn_route_plan.rs and follow SavedSession.last_auto_route through crates/tui/src/session_manager.rs and its writes in tui/ui/event_loop.rs and dispatch.rs. Review the dispatch and re-resolution paths in client.rs and route/resolver.rs, then verify the listed acceptance cases, including StickyBinding routing, credential-triggered rebinds, cache stability, and explicit model changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai, cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.