electric-sql / electric-sql/electric

Live SSE short-connection fallback triggered by CDN cache-collapsed responses

Open
#4,137 1 comment 3 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
10.4k
Forks
375
Avg merge
3d 1h
Merged PRs (30d)
18

Description

When using `liveSse: true` against Electric **Cloud**, the client repeatedly logs `[Electric] SSE connections are closing immediately (possibly due to proxy buffering or misconfiguration). Falling back to long polling.` and falls back to long polling. Instrumentation shows the underlying SSE responses are actually healthy cache-collapsed responses served by the CDN in front of Electric Cloud's origin — not proxy buffering. Data still syncs via the long-polling fallback, but the warnings are noisy and suggest a misconfiguration that doesn't exist.

- `@electric-sql/client@1.5.15`
- Backend: Electric **Cloud** (not self-hosted). Reproduces identically against `api.electric-sql.cloud` with a plain `curl`.
- Client: Chromium browser; also reproducible from curl.

**Setup**

Our Bun server auth-proxies `/api/sync/*` → `api.electric-sql.cloud/v1/shape` following the pattern in the [Cloudflare integration docs](https://electric-sql.com/docs/integrations/cloudflare). The proxy is a faithful `fetch()` → `new Response(upstream.body, upstream)` passthrough, instrumented to log every chunk, EOF, error, and cancel on the upstream body.

**Observed behavior**

Each live SSE response falls into one of two cleanly-bimodal patterns:

| Pattern | Chunks | Bytes | Duration | Who closes the stream? |
| ------- | -------------------------------- | ----------------------------------------- | -------- | ---------------------- |
| Short | 1 | 28 (two stitched `: keep-alive` comments) | 30–75ms | Upstream EOF |
| Long | 2 (keep-alives at ~21s and ~42s) | 28 | ~60s | Upstream EOF |

Every EOF originates upstream — the proxy never cancels, the client's abort signal never fires. The "long" timing matches `sse_timeout: 60_000`, `keepalive_interval: 21_000`, `max_age: 60` in `packages/sync-service/lib/electric/shapes/api.ex`.

**Reproducer**

Bypasses our proxy entirely:

```bash
# 1. initial snapshot → grab the handle from the electric-handle header
curl -sNi "https://api.electric-sql.cloud/v1/shape?table=TABLE&source_id=SRC&secret=SECRET&offset=-1" | grep -i electric-handle

# 2. live SSE — run N times, observe bimodal durations
curl -N -sS -w 'total=%{time_total}s size=%{size_download}\n' \
"https://api.electric-sql.cloud/v1/shape?table=TABLE&source_id=SRC&secret=SECRET&handle=$HANDLE&live=true&live_sse=true&log=full&offset=0_0&experimental_live_sse=true" \
-H 'Accept: text/event-stream' --max-time 30
```

Most runs complete in ~0.15s with 28 bytes; occasional runs hold for the full 60s.

**Does not reproduce** against a locally-dockerized Electric (no CDN between client and origin) — every request gets a fresh 60s stream.

**Hypothesis**

Electric serves live SSE responses with `Cache-Control: public, max-age=60`. The CDN in front of Electric Cloud's origin appears to serve cache-collapsed responses to subsequent subscribers within that 60s window; those collapsed bodies are just "two keep-alive comments + EOF", and they complete in well under 1s.

The TypeScript client's short-connection heuristic in `LiveState.handleSseConnectionClosed` (`packages/typescript-client/src/shape-stream-state.ts`) decides purely on duration — the `SseCloseInput` interface contains only `{ connectionDuration, wasAborted, minConnectionDuration, maxShortConnections }`, with no chunk count, no response headers, no `hasData` flag. So when a cache-collapsed response returns cleanly in <1s, the method has no way to distinguish "healthy cache hit that delivered the expected keep-alives + EOF" from "connection killed by a buffering proxy before anything flowed" — both look like `connectionDuration < 1000 && !wasAborted`. After three such closures in a row the shape falls back to long polling and logs the proxy-buffering warning, even though nothing is actually buffered.

The "only in prod" shape of the bug fits this — local dev has no CDN between client and origin, so there's nothing to collapse and every SSE connection easily clears the 1s threshold.

**Related**

- #3635 — feature request to make `sse_timeout` configurable; describes the same 60s / 21s timing and guesses request-collapsing is involved.
- #3816 — recent `ShapeStream` state-machine refactor (merged); introduces the `handleSseConnectionClosed` call site under discussion here.
- #4015 — stale-CDN infinite-loop fix (merged); adjacent "client doesn't handle CDN behavior correctly" bug, different mechanism.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.