No retry/backoff for transient API errors (429, 5xx) in OpenAI-compatible provider runtime

Summary

The OpenAI-compatible provider runtime (used by glm-coding, OpenRouter, DeepSeek, and all type = "openai-compatible" profiles) only retries transient errors 3 times with ~1s/2s/4s exponential backoff. Under sustained rate limiting (HTTP 429) or provider overload — especially with concurrent swarm agents sharing one endpoint — all 3 attempts fail within ~9 seconds and the turn stops. This is notably worse than opencode, which backs off and keeps retrying until the condition clears.

The retry budget and backoff interval are hardcoded constants (MAX_RETRIES = 3, RETRY_BASE_DELAY_MS = 1000) with no user-facing configuration.

Reproduction

1. Configure an OpenAI-compatible provider that rate-limits frequently (e.g. Zhipu/ZAI glm-coding-plan at open.bigmodel.cn):

   [providers.glm-coding]
   type = "openai-compatible"
   base_url = "https://open.bigmodel.cn/api/coding/paas/v4"
   

2. Run a swarm with several concurrent agents, or send rapid sequential requests.
3. Observe the provider returning 429 Too Many Requests ({"error":{"code":"1302","message":"Rate limit reached for requests"}}).

Actual behavior

• The runtime retries 3 times with ~1s/2s/4s backoff (~9 seconds total).
• After exhausting retries, the turn fails with stream_error and the session stops.
• The user must manually resend the message.
• No ConnectionPhase::Retrying event is emitted to the TUI, so the user sees no feedback during the brief retry window.

Log evidence:

[ERROR] AGENT_PROVIDER_STREAM_LIFECYCLE ... error="OpenAI-compatible chat request failed
  endpoint: https://open.bigmodel.cn/api/coding/paas/v4/chat/completions
  model: glm-5.2  status: 429 Too Many Requests
  response: {\"error\":{\"code\":\"1302\",\"message\":\"Rate limit reached for requests\"}}"
  phase=stream_error

Expected behavior

• The retry budget and backoff cap should be configurable via [provider] in config.toml, following the same pattern as the existing stream_idle_timeout_secs option.
• Defaults should be more resilient (~5 minutes of retrying) while remaining bounded so a genuine outage doesn't hang the turn indefinitely.
• The TUI should show a retrying status (ConnectionPhase::Retrying) so the user knows the agent is backing off, not frozen.
• Server Retry-After headers should still be honored.

Root cause

In crates/jcode-provider-openrouter-runtime/src/lib.rs:

const MAX_RETRIES: u32 = 3;
const RETRY_BASE_DELAY_MS: u64 = 1000;

These are compile-time constants with no config or env-var override. The retry loop in openrouter_sse_stream.rs:run_stream_with_retries uses them directly. The loop also does not emit any ConnectionPhase event to communicate retry status to the consumer.

The same hardcoded MAX_RETRIES = 3 pattern exists in the Anthropic and native OpenAI runtimes as well.

Proposed fix

• Add max_retries and retry_backoff_cap_secs fields to ProviderConfig (in jcode-config-types), following the stream_idle_timeout_secs pattern.
• Add jcode_base::provider::{max_retries, retry_backoff_cap} resolver functions.
• Add JCODE_MAX_RETRIES / JCODE_RETRY_BACKOFF_CAP_SECS env-var overrides.
• Update run_stream_with_retries to use the config resolvers, cap the exponential backoff, and emit ConnectionPhase::Retrying { attempt, max } for TUI visibility.
• Suggested defaults: max_retries = 8, retry_backoff_cap_secs = 30 (~5 minutes of retrying). Users who hit frequent throttling can raise these (e.g. 30/60 for ~20 minutes).

I have a working implementation ready as a PR (blocked by repo fork-PR permissions — branch is at Jichao:jcode:feat/configurable-transient-error-retries).

Related

• The Anthropic runtime (jcode-provider-anthropic-runtime/src/lib.rs) and native OpenAI runtime (jcode-provider-openai-runtime/src/openai_provider_impl.rs) share the same MAX_RETRIES = 3 limitation and would benefit from the same config-driven approach.
• The outer cross-provider failover (jcode-base/src/provider/failover.rs) handles switching providers, but when only one provider is configured (common for coding-plan endpoints), there is nothing to fail over to — making per-provider retry resilience critical.

Start in crates/jcode-provider-openrouter-runtime/src/lib.rs and openrouter_sse_stream.rs:run_stream_with_retries, then trace ProviderConfig in jcode-config-types and the provider resolver functions in jcode-base. Done means configurable bounded retries and backoff across the named runtimes, Retry-After support, and ConnectionPhase::Retrying visibility in the TUI.