anomalyco / anomalyco/opencode

[FEATURE]: Configurable model fallback on transient errors and timeouts

Open
#48,991 1 comment 0 reactions 1 assignee View on GitHub

@neriousy is already working on this.

Since Sep 14, 2026.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Feature hasn't been suggested before.
  • I have verified this feature I'm about to request hasn't been suggested before.
Describe the enhancement you want to request

Supersedes #7602 (Jan 2026) and the closed PR #20105 (Mar 2026, auto-cleaned). Incorporates lessons learned from #20105 and aligns with the current Effect-based session architecture.

Problem

When a model is unreachable, rate-limited, or times out, the session retry policy (session/retry.ts) retries the same model on the same provider with exponential backoff. After retries are exhausted, halt() is called and the session dies. There is no mechanism to fall back to a different model.

This causes long-running agent workflows to fail on transient provider issues that a different provider would handle fine.

Proposed Behavior

After the retry policy exhausts for a transient error (5xx, 429, network failure, timeout), instead of halting the session:

  1. Look up the configured fallback model(s) for the current model
  2. If a fallback exists and is available, switch to it and re-run the LLM stream
  3. Track which fallback was used for status reporting
  4. Emit a status event so the UI can show the model switch
  5. If the fallback also fails, try the next in the chain
  6. If all fallbacks are exhausted, halt as before

Triggers

  • Rate limits (429, Too Many Requests)
  • Server errors (500, 502, 503)
  • Network failures (ECONNREFUSED, ECONNRESET, ETIMEDOUT, fetch failed)
  • Timeouts (header timeout, stream timeout — the primary use case)
  • Model unavailable (404, model not found)

Not triggered by:

  • Auth errors (401 — different provider has different creds)
  • Context overflow (413 — needs compaction, not a model switch)
  • Validation/prompt errors

Config Schema

Aligns with the existing provider.models config structure:

{
  "provider": {
    "anthropic": {
      "models": {
        "claude-sonnet-4": {
          "fallback": ["openai/gpt-5", "google/gemini-3-pro"]
        }
      }
    },
    "openai": {
      "models": {
        "gpt-5": {
          "fallback": ["anthropic/claude-sonnet-4", "google/gemini-3-pro"]
        }
      }
    }
  }
}

Each entry in the fallback array is a "provider/model-id" string. The array defines an ordered fallback chain — tried in sequence until one succeeds.

Architectural Notes

PR #20105 implemented fallback as LLM middleware via wrapLanguageModel. This approach has two issues:

  1. Interacts badly with the session retry loop — middleware runs inside the AI SDK's own retry, which conflicts with the Effect-based SessionRetry.policy
  2. Only 1 fallback — hardcoded Copilot-to-Bedrock mapping, not a configurable chain

The recommended approach is to implement fallback at the session processor layer (processor.ts), after Effect.retry(SessionRetry.policy(...)) exhausts:

Effect.retry(SessionRetry.policy(...))
  -> Effect.catch(fallback)   // instead of Effect.catch(halt)
  -> Effect.catch(halt)       // if all fallbacks exhausted

This keeps fallback logic in the Effect-based session pipeline where the error context is richest.

References

  • #7602 — Original feature request (Jan 2026)
  • #20105 — Closed PR implementing Copilot-to-Bedrock fallback (Mar 2026, auto-cleaned)
  • #42513 — Related: API key pooling per provider
  • #43324 — Related: Per-provider quota-aware retry

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.