Per-tool model routing: delegate lightweight tools to a cheaper/faster secondary model
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 19.9k
- Forks
- 2.3k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 30
Description
Problem
When using a paid API plan (e.g. Z.AI Coding Plan, Claude Max, OpenAI), all tool calls consume the same model quota regardless of complexity. A web search, file grep, or browser action costs the same credits as complex reasoning or multi-file refactoring. There is no way to route lightweight tools to a cheaper/faster model while keeping the flagship model for hard tasks.
Concrete example with Z.AI Coding Plan credits:
| Tool type | Typical token cost | Model needed |
|---|---|---|
| File search / grep | 2k-10k tokens | Any |
| Web search | 1k-5k tokens | Any |
| Browser navigation | 2k-10k tokens | Any |
| Multi-file refactor | 50k-200k tokens | Flagship |
| Complex debugging | 50k-100k tokens | Flagship |
Routing the first three rows to GLM-4.7 instead of GLM-5.2 saves ~15% credits per call on Z.AI. Over a week of agentic coding, this can meaningfully extend the weekly quota.
Proposed feature
Add a [[tools.model_override]] config table that routes specific tools to a different provider/model:
# Default model for all tools
# (set via provider/model selection as today)
# Override: route specific lightweight tools to a cheaper model
[[tools.model_override]]
tools = ["agentgrep", "websearch", "webfetch", "browser"]
provider = "zai-coding-plan"
model = "GLM-4.7"
# Keep heavy tools on the flagship
# (implicitly uses the default model)
Behavior
- When the agent decides to call a tool in the override list, that single tool call is sent to the overridden model/provider instead of the session model.
- Tool call inputs and outputs are still injected into the main conversation history for the primary model.
- The override applies to the tool execution itself (choosing parameters, interpreting results), not to the model deciding which tool to call.
- Falls back to the primary model if the override provider is unavailable.
Edge cases to consider
- Should MCP tool calls be overridable separately?
- Should there be a per-override concurrency limit to avoid hammering the cheaper provider?
- How does this interact with compaction (which model summarizes)?
- Swarm workers already support per-agent model selection. This feature is the single-agent analogue.
Alternatives considered
- Manual model switching (Ctrl+Tab): Works but tedious and error-prone. Users forget to switch back.
- Swarm delegation: Works for bulk search tasks but overkill for a single web search. Adds latency and complexity.
- Prompt engineering: Asking the model to "use tools sparingly" reduces calls but does not change per-call cost.
Who benefits
Any user on a metered or flat-fee API plan who wants to maximize their quota by reserving expensive model capacity for tasks that actually need it. Especially relevant for Z.AI Coding Plan users (Lite/Pro/Max) and Claude Max users who hit weekly caps.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points; start by locating the current provider/model selection and tool-execution paths. Define the configuration and behavior around per-tool overrides, fallback handling, conversation history, and the listed edge cases, then verify that lightweight tools use the override while the primary model handles the rest.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ai, cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100