SSE keepalive uses a comment frame, which cannot reset OpenAI Codex's idle timeout (only the messages path sends a data frame)

Filed by Claude Code (Anthropic's CLI), on behalf of a Floway operator, after a source-level investigation of a production deployment. Every code reference below was read out of the running 20260816.31959732430 image or fetched from the upstream repo of the client/crate in question.

Summary

Floway keeps a quiet downstream connection alive by writing an SSE comment frame (: keepalive). OpenAI Codex — the primary consumer of /responses — parses SSE with the eventsource-stream crate, which discards comment lines and yields no item to the consumer. Codex's idle timer wraps stream.next(), so Floway's keepalive can never reset it.

The result is a silent, total failure on a connection that is completely healthy: neither side closes it, bytes really are flowing every 15 s, and the client still gives up after stream_idle_timeout_ms (default 300_000) with:

stream disconnected before completion: idle timeout waiting for SSE

Floway's own Anthropic messages path already does the right thing — it emits a data-bearing ping event. It is the only one of the six streaming paths that does.

The inconsistency

packages/gateway/src/data-plane/chat/messages/respond.ts:72
  keepAlive: { frame: sseFrame(JSON.stringify({ type: 'ping' }), 'ping') }   ← data frame ✅

packages/gateway/src/data-plane/chat/responses/respond.ts:79
packages/gateway/src/data-plane/chat/chat-completions/respond.ts:69
packages/gateway/src/data-plane/chat/gemini/respond.ts:72
packages/gateway/src/data-plane/audio/respond.ts:68
packages/gateway/src/data-plane/shared/passthrough-serve.ts:224
  keepAlive: { frame: sseCommentFrame('keepalive') }                          ← comment frame ❌

Interval is DOWNSTREAM_KEEP_ALIVE_INTERVAL_MS = 15_000 (packages/gateway/src/data-plane/shared/sse.ts:5).

Why the client cannot see it

eventsource-stream, src/event_stream.rs:

RawEventLine::Comment(_) => {}

The blank line terminating the frame does set is_complete, but dispatch() returns None because the data buffer is empty, so the poll loop continues without producing an item.

codex-rs, codex-rs/codex-api/src/sse/responses.rs:

let response = timeout(idle_timeout, stream.next()).await;
...
Err(_) => {
    let _ = tx_event
        .send(Err(ApiError::Stream("idle timeout waiting for SSE".into())))
        .await;
    return;
}

No item ⇒ no timer reset. Default DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000.

Why this bites hardest on Responses + Copilot

Floway's own source already documents that Copilot Responses streams return HTTP 200 and then go silent for minutes before the first token — packages/gateway/src/dial/fetcher.ts explains that direct egress deliberately uses a raw socket instead of fetch because "measured silences run past both bounds (120s and 300.113s observed on the same workload)", citing #221 where a workload "survived 233s of measured upstream silence and completed cleanly".

So Floway is right to hold the socket open — but the frame it sends during exactly that window is invisible to the client that needs it most.

From the deployment I investigated (Floway's own performance_buckets, metric = 'ttft_ms', model LIKE 'gpt-5.6-sol%', 443,108 samples):

TTFT bucket	samples
250,000 – 500,000 ms	354
500,000 – 1,000,000 ms	32
1,000,000 – 2,500,000 ms	8

These are successful samples — the first token did eventually arrive. Requests where Codex gave up first never record a TTFT sample at all (they fall into neutral), so the real incidence of >300 s silence is higher than the table can show.

Suggested fix

Mirror the messages path:

keepAlive: { frame: sseFrame(JSON.stringify({ type: 'keepalive' }), 'keepalive') },

This is safe for Codex — both of its tolerance paths are explicit (codex-rs/codex-api/src/sse/responses.rs):

// unrecognised event type
_ => {
    debug!("unhandled responses event: {:?}", event.kind.chars().take(128).collect::<String>());
}

// unparseable data
Err(e) => {
    debug!(error_category = ?e.classify(), ..., "Failed to parse SSE event");
    continue;
}

Either way the item reaches the consumer and the idle timer resets. The essential requirement is only that the frame carry a data: line — the payload itself is not load-bearing.

Secondary: the shim drops data-bearing heartbeats the upstream did send

packages/gateway/src/data-plane/chat/responses/interceptors/server-tool-shim.ts:660

if ((event.type as string) === 'ping' || (event.type as string) === 'keepalive') continue;

Dropping these on the way downstream removes real heartbeats that would have reset the client's timer. Filtering them from the upstream-facing accumulator makes sense; suppressing them downstream compounds the problem above.

Environment

• ghcr.io/menci/floway-server:20260816.31959732430 (web same tag)
• Upstream: GitHub Copilot (api.enterprise.githubcopilot.com), gpt-5.6-sol-fast, reasoning effort ultra
• Client: OpenAI Codex CLI and Codex Desktop 0.150.0-alpha, wire_api = "responses"
• Reverse proxy: Caddy with flush_interval -1; floway-web nginx with proxy_buffering off

Ruled out during the investigation

undici bodyTimeout (Floway's raw-socket egress deliberately avoids it); downstream buffering (verified at both Caddy and nginx); server OOM/restarts (RestartCount 0 over 12 days, no OOM in dmesg); the network path (measured clean end to end from three vantage points, including a connection that survived 300 s idle and still returned 200); and upstream mid-stream errors — errors_with_output = 0 across 21,066 requests, i.e. no stream that had already produced a first token ever failed. 100% of failures occur before the first token, which is precisely the window the keepalive is supposed to cover.

Start with the keepAlive definitions in packages/gateway/src/data-plane/chat/responses/respond.ts and the other paths listed in the issue, then read packages/gateway/src/data-plane/shared/sse.ts for the interval. Inspect server-tool-shim.ts around line 660 as well. Done means downstream SSE keepalives carry data and are not discarded before reaching clients such as Codex during long pre-token silences.