anomalyco / anomalyco/opencode

[FEATURE]:Auto-Failover for Rate Limit Resilience

Open
#42,513 2 comments 2 reactions 1 assignee View on GitHub

@jlongster is already working on this.

Since Aug 14, 2026.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Been hitting this nonstop on longer refactoring and large codebase work — OpenCode only takes one API key per provider right now, and the second that key hits a 429 rate limit, runs into a random 500 server error, or burns through daily quota, the whole agent session dies halfway through. You have to manually swap keys and restart everything, which completely defeats the point of letting it run unattended.

I know issue #7602 covers model-level fallback, but this is a separate, narrower request. That one is about falling back to different model types when a provider fails; this is about pooling multiple keys for the same provider and same model to spread out request load and fail over instantly when a single key gets throttled. No manual intervention, no lost progress. They complement each other, not overlap.

The core ask is simple: build key pooling natively into the LLM adapter layer. Let users stack keys via numbered environment variables, like OPENAI_KEY_1, OPENAI_KEY_2, and cycle through them round-robin by default to even out request pressure. If a request comes back 429, immediately retry that exact same call on the next available key instead of throwing an error and killing the task. Honor the Retry-After header when the provider sends it, otherwise stick the throttled key into a 60-second cooldown before putting it back in rotation.

Error handling can stay straightforward, no need for anything overengineered. 5xx server errors trigger a short cooldown after a couple consecutive failures so it doesn't hammer a broken endpoint. 401/403 auth failures mark the key as invalid entirely, stop routing traffic to it, and show a clear notice in the UI. Keys that hit daily quota limits stay disabled until UTC midnight reset.

No breaking changes either. If you only have one key configured, everything behaves exactly like it does today. Would also be nice to tuck a tiny status indicator in the TUI footer — show the active key masked to its last 4 characters, and how many are sitting in cooldown — just so you can tell what's going on at a glance.

I'm keeping this intentionally narrow, focused solely on key pooling and failover logic. All the other ideas I brought up earlier around caching and repo structure I'll file as separate focused issues later. Just want to fix the rate-limit crash problem first, since it's the most consistent day-to-day frustration.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.