GcsSloop / GcsSloop/ai-gate

Request for improving /v1/responses soft-failure diagnostics and pre-output failover for official Codex OAuth accounts

Open
#123 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
44
Forks
4
PR merge metrics
No merged PRs in 30d

Description

## Summary

`/v1/responses` sometimes returns:

```json
{"error":{"code":"responses_unsupported","message":"no healthy responses account available","type":"invalid_request_error"}}
```

This happens even when the active official OpenAI/Codex OAuth account still has remaining quota and is marked active/
supporting responses.

## Environment

- OS: Windows + WSL
- Client: Codex CLI
- ai-gate mode: local router / thin gateway
- Codex config:
- base_url = "http://127.0.0.1:6789/ai-router/api"
- wire_api = "responses"
- Upstream proxy: Clash Verge / mihomo
- Account setup:
- one official OpenAI/Codex OAuth imported account
- optional OpenAI-compatible fallback accounts
- Model: gpt-5.5
- Request type: streaming /v1/responses

## Observed Behavior

The router intermittently returns responses_unsupported/no healthy responses account available.

From the local usage_events table, the official OAuth account shows many soft_failed events with zero tokens, often
after long latency.

Example aggregate from a pre-restore DB snapshot:

account_id=1 openai-official responses gpt-5.5 completed 595 avg_latency≈17s max≈260s
account_id=1 openai-official responses gpt-5.5 soft_failed 203 avg_latency≈71s max≈907s

Recent examples:

status=soft_failed input_tokens=0 output_tokens=0 latency≈34s
status=soft_failed input_tokens=0 output_tokens=0 latency≈24s
status=soft_failed input_tokens=0 output_tokens=0 latency≈46s

The account was not disabled, not invalid, not locked, and not obviously exhausted. Usage snapshot still showed
remaining quota.

## Why This Is Hard To Diagnose

usage_events.status = soft_failed is too coarse. It is not clear whether the failure occurred at:

- session refresh / ensureOfficialAccountSession
- upstream request creation
- upstream HTTP status
- streaming read
- EOF before response.completed
- timeout
- proxy/network transport error
- official Codex backend error

The final client-facing error responses_unsupported also seems misleading when the account does support /responses;
the actual problem appears to be that no healthy candidate survived routing/failover.

## Expected Behavior

1. If an account supports /responses but the upstream stream/request fails, the returned error should distinguish this
from unsupported responses.
2. usage_events or another local diagnostic table/log should include failure stage and compact error reason.
3. If failure happens before any response bytes are streamed to the client, ai-gate should retry/fail over more
robustly.
4. If fallback accounts are available, the router should log from-account/to-account failover decisions clearly.

## Suggested Improvements

### Diagnostics

Add structured fields to usage/logs, for example:

failure_stage = ensure_session | upstream_request | upstream_status | read_stream
error_class = soft | rate_limit | capacity | hard
error_reason = eof | timeout | stream_closed_before_response_completed | http_429 | http_403 | upstream_5xx | unknown
from_account_id
failover_succeeded
request_input_token_estimate or request_body_size

### Behavior

For official Codex OAuth /responses streaming:

- retry EOF/timeout/transport errors with short backoff before returning to Codex
- if no bytes have been written to the client, attempt failover to the next healthy /responses account
- consider a temporary circuit breaker for accounts with repeated soft_failed events
- return a more accurate error code than responses_unsupported when the account supports /responses but all candidates
failed

## Notes

I understand ai-gate is intentionally a thin gateway and cannot safely replay a stream after partial output has
already been sent to the client. The main request here is better pre-output retry/failover and better observability
for soft_failed, so users can distinguish quota, upstream instability, stream EOF, timeout, and routing problems.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the /v1/responses streaming path, ensureOfficialAccountSession, and usage_events records to identify where soft failures are classified. Define completion as stage-specific diagnostics, accurate errors, and pre-output failover with clear account transition logs, while preserving the stated limitation on retrying after output begins.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend, databases, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.