1jehuang / 1jehuang/jcode

Per-tool model routing: delegate lightweight tools to a cheaper/faster secondary model

Open
#810 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

autonomous: no enhancement triage: needs-decision
Dominant language
Rust
Stars
19.9k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
30

Description

Problem

When using a paid API plan (e.g. Z.AI Coding Plan, Claude Max, OpenAI), all tool calls consume the same model quota regardless of complexity. A web search, file grep, or browser action costs the same credits as complex reasoning or multi-file refactoring. There is no way to route lightweight tools to a cheaper/faster model while keeping the flagship model for hard tasks.

Concrete example with Z.AI Coding Plan credits:

Tool type Typical token cost Model needed
File search / grep 2k-10k tokens Any
Web search 1k-5k tokens Any
Browser navigation 2k-10k tokens Any
Multi-file refactor 50k-200k tokens Flagship
Complex debugging 50k-100k tokens Flagship

Routing the first three rows to GLM-4.7 instead of GLM-5.2 saves ~15% credits per call on Z.AI. Over a week of agentic coding, this can meaningfully extend the weekly quota.

Proposed feature

Add a [[tools.model_override]] config table that routes specific tools to a different provider/model:

# Default model for all tools
# (set via provider/model selection as today)

# Override: route specific lightweight tools to a cheaper model
[[tools.model_override]]
tools = ["agentgrep", "websearch", "webfetch", "browser"]
provider = "zai-coding-plan"
model = "GLM-4.7"

# Keep heavy tools on the flagship
# (implicitly uses the default model)
Behavior
  • When the agent decides to call a tool in the override list, that single tool call is sent to the overridden model/provider instead of the session model.
  • Tool call inputs and outputs are still injected into the main conversation history for the primary model.
  • The override applies to the tool execution itself (choosing parameters, interpreting results), not to the model deciding which tool to call.
  • Falls back to the primary model if the override provider is unavailable.
Edge cases to consider
  • Should MCP tool calls be overridable separately?
  • Should there be a per-override concurrency limit to avoid hammering the cheaper provider?
  • How does this interact with compaction (which model summarizes)?
  • Swarm workers already support per-agent model selection. This feature is the single-agent analogue.

Alternatives considered

  1. Manual model switching (Ctrl+Tab): Works but tedious and error-prone. Users forget to switch back.
  2. Swarm delegation: Works for bulk search tasks but overkill for a single web search. Adds latency and complexity.
  3. Prompt engineering: Asking the model to "use tools sparingly" reduces calls but does not change per-call cost.

Who benefits

Any user on a metered or flat-fee API plan who wants to maximize their quota by reserving expensive model capacity for tasks that actually need it. Especially relevant for Z.AI Coding Plan users (Lite/Pro/Max) and Claude Max users who hit weekly caps.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points; start by locating the current provider/model selection and tool-execution paths. Define the configuration and behavior around per-tool overrides, fallback handling, conversation history, and the listed edge cases, then verify that lightweight tools use the override while the primary model handles the rest.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai, cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.