Request for improving /v1/responses soft-failure diagnostics and pre-output failover for official Codex OAuth accounts
- Dominant language
- Go
- Stars
- 44
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
`/v1/responses` sometimes returns:
```json
{"error":{"code":"responses_unsupported","message":"no healthy responses account available","type":"invalid_request_error"}}
```
This happens even when the active official OpenAI/Codex OAuth account still has remaining quota and is marked active/
supporting responses.
## Environment
- OS: Windows + WSL
- Client: Codex CLI
- ai-gate mode: local router / thin gateway
- Codex config:
- base_url = "http://127.0.0.1:6789/ai-router/api"
- wire_api = "responses"
- Upstream proxy: Clash Verge / mihomo
- Account setup:
- one official OpenAI/Codex OAuth imported account
- optional OpenAI-compatible fallback accounts
- Model: gpt-5.5
- Request type: streaming /v1/responses
## Observed Behavior
The router intermittently returns responses_unsupported/no healthy responses account available.
From the local usage_events table, the official OAuth account shows many soft_failed events with zero tokens, often
after long latency.
Example aggregate from a pre-restore DB snapshot:
account_id=1 openai-official responses gpt-5.5 completed 595 avg_latency≈17s max≈260s
account_id=1 openai-official responses gpt-5.5 soft_failed 203 avg_latency≈71s max≈907s
Recent examples:
status=soft_failed input_tokens=0 output_tokens=0 latency≈34s
status=soft_failed input_tokens=0 output_tokens=0 latency≈24s
status=soft_failed input_tokens=0 output_tokens=0 latency≈46s
The account was not disabled, not invalid, not locked, and not obviously exhausted. Usage snapshot still showed
remaining quota.
## Why This Is Hard To Diagnose
usage_events.status = soft_failed is too coarse. It is not clear whether the failure occurred at:
- session refresh / ensureOfficialAccountSession
- upstream request creation
- upstream HTTP status
- streaming read
- EOF before response.completed
- timeout
- proxy/network transport error
- official Codex backend error
The final client-facing error responses_unsupported also seems misleading when the account does support /responses;
the actual problem appears to be that no healthy candidate survived routing/failover.
## Expected Behavior
1. If an account supports /responses but the upstream stream/request fails, the returned error should distinguish this
from unsupported responses.
2. usage_events or another local diagnostic table/log should include failure stage and compact error reason.
3. If failure happens before any response bytes are streamed to the client, ai-gate should retry/fail over more
robustly.
4. If fallback accounts are available, the router should log from-account/to-account failover decisions clearly.
## Suggested Improvements
### Diagnostics
Add structured fields to usage/logs, for example:
failure_stage = ensure_session | upstream_request | upstream_status | read_stream
error_class = soft | rate_limit | capacity | hard
error_reason = eof | timeout | stream_closed_before_response_completed | http_429 | http_403 | upstream_5xx | unknown
from_account_id
failover_succeeded
request_input_token_estimate or request_body_size
### Behavior
For official Codex OAuth /responses streaming:
- retry EOF/timeout/transport errors with short backoff before returning to Codex
- if no bytes have been written to the client, attempt failover to the next healthy /responses account
- consider a temporary circuit breaker for accounts with repeated soft_failed events
- return a more accurate error code than responses_unsupported when the account supports /responses but all candidates
failed
## Notes
I understand ai-gate is intentionally a thin gateway and cannot safely replay a stream after partial output has
already been sent to the client. The main request here is better pre-output retry/failover and better observability
for soft_failed, so users can distinguish quota, upstream instability, stream EOF, timeout, and routing problems.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the /v1/responses streaming path, ensureOfficialAccountSession, and usage_events records to identify where soft failures are classified. Define completion as stage-specific diagnostics, accurate errors, and pre-output failover with clear account transition logs, while preserving the stated limitation on retrying after output begins.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend, databases, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100