1jehuang / 1jehuang/jcode

auto provider selection has no way to express provider preference or remaining quota

Open
#950 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

autonomous: no enhancement priority: medium triage: needs-decision
Dominant language
Rust
Stars
19.9k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
30

Description

Summary

auto picks a provider from a fixed if/else, and the struct it reads carries no notion of cost or remaining quota. For users with several subscriptions configured, that means every session opens against a provider that may be exhausted, spends a request discovering it, and only then fails over.

Where

crates/jcode-provider-core/src/selection.rs, auto_default_provider (~L44-60):

if availability.copilot_premium_zero && availability.copilot {
    ActiveProvider::Copilot
} else if availability.claude {
    ActiveProvider::Claude
} else if availability.openai {
    ActiveProvider::OpenAI
} else if availability.copilot {
    ActiveProvider::Copilot
} ...

Its only input is ProviderAvailability (~L16-27), which is booleans plus copilot_premium_zero — "is this provider configured", never "does it have headroom".

Why it bites

Concrete case: ChatGPT window at 100% with days to reset, Copilot with plenty left, both authenticated. auto resolves to OpenAI (position 3) ahead of Copilot (position 4) every time. cross_provider_failover then does its job correctly and moves to Copilot — but only after a failed request plus the countdown, once per session.

The failover path is reactive by design (failover.rs classifies 429 / quota / 402 / context-length), which is right for a provider that dies mid-session. It just can't route around one already known to be spent.

copilot_premium_zero is the only ordering lever today, and it's really a model-tier restriction that happens to reorder as a side effect.

Suggestion

An explicit order in config, falling back to the current chain when unset:

[provider]
# first configured provider in this list wins; unlisted fall back to the built-in chain
order = ["copilot", "claude", "openai"]

That's a small, backwards-compatible change: auto_default_provider takes the list and returns the first entry where is_configured holds, else current behaviour.

A richer version could skip providers already known to be rate-limited this session — the failover code already learns that (RetryAndMarkUnavailable), so the signal exists; it just isn't consulted at selection time.

Workaround

For anyone hitting this now: jcode provider list + jcode usage before launch, then -p <chosen>. -p is honoured on launch, and jcode provider current conveniently prints requested_provider / resolved_provider / selected_model so the choice can be confirmed rather than assumed.

Happy to send a PR for the order key if the approach seems reasonable.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in crates/jcode-provider-core/src/selection.rs, especially ProviderAvailability and auto_default_provider around lines 16-60. Read the related failover behavior in failover.rs, including RetryAndMarkUnavailable, then trace provider configuration handling. Done means an optional provider order selects the first configured provider and preserves the current chain when unset.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.