electric-sql / electric-sql/electric
RFC: Websocket wake multiplexing
- Dominant language
- TypeScript
- Stars
- 10.4k
- Forks
- 375
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 18
Description
## Motivation
A live shape request currently retains roughly 50–80 KB while it waits. At 200,000 concurrent readers this can consume around 10 GB even when almost every shape is silent; in the observed worst case only about 1,000 readers receive changes during a polling interval.
Add an authenticated, internal WebSocket endpoint that multiplexes only the waiting part of live requests. Shape data must continue over the ordinary HTTP shape endpoint so cache behavior, streaming, and client compatibility remain unchanged.
## Proposed endpoint
`GET /v1/shape/multiplex`, using the same source selection and service-secret authentication as `/v1/shape`.
The endpoint upgrades only on the active Electric instance. An inactive, read-only, or non-owner instance rejects the upgrade retryably. An established socket closes if that instance stops being active.
Protocol identifier: `electric.shape-multiplex.v1`.
Client frames:
```json
{"type":"watch","id":"opaque-request-id","handle":"shape-handle","offset":"123_4","cursor":"456"}
{"type":"unwatch","id":"opaque-request-id"}
```
`cursor` is the raw, unwrapped cursor from the preceding Electric response. Electric uses it to generate a strictly advancing response cursor with the existing live-response algorithm, preventing a cache from replaying the same empty interval.
A watch is level-triggered: Electric registers for shape events and then rechecks the current head, so a change racing with registration produces an immediate wake. Handles must already exist; handle/offset pairs receive lightweight validation. The canonical post-snapshot `0_inf` offset is eligible (including never-written shapes); initial `-1`, `now`, and other special offsets are not.
Electric sends at most one terminal frame for each watch:
```json
{"type":"wake","id":"opaque-request-id","reason":"changes"}
{"type":"wake","id":"opaque-request-id","reason":"rotation"}
{"type":"no_change","id":"opaque-request-id","response":{"status":200,"headers":{"electric-cursor":"...","electric-handle":"...","electric-offset":"...","electric-has-data":"false","cache-control":"...","etag":"..."},"body":[{"headers":{"control":"up-to-date","global_last_seen_lsn":"..."}}]}}
{"type":"error","id":"opaque-request-id","code":"...","message":"...","retryable":true}
```
A `ready` acknowledgement may be sent after registration. Unknown fields are ignored for forward compatibility.
## Deadline and lifecycle semantics
Electric owns the deadline, using the normal configured live-request timeout. The caller cannot extend it. At the deadline Electric constructs the same no-change semantics as an HTTP live response, sends `no_change`, and removes the logical watch. Data/rotation, cancellation, socket closure, and errors also remove it. Registry subscriptions should exist only while a socket has logical watches for the handle.
This deliberately avoids a follow-up HTTP probe on timeout. The unchanged client immediately issues its next live request; a new multiplex watch performs the subscribe-then-head recheck and wakes immediately if data appeared after the preceding deadline.
## Proxy integration
Existing clients need no change. A proxy can encode the direct/multiplex phase in the opaque `electric-cursor` it already returns:
1. Serve the first live request directly over HTTP.
2. If it returns `electric-has-data: false`, wrap the next cursor with a versioned quiet marker.
3. A marked request waits through the multiplex connection after unwrapping and forwarding its raw cursor.
4. `no_change` becomes the ordinary HTTP no-change response and stays marked.
5. `wake` causes a cache-busted direct HTTP request to Electric; the response body never passes through the WebSocket coordinator and the cursor returns to direct mode.
6. Socket refusal/failure tells the proxy to use ordinary HTTP. The coordinator itself never performs HTTP fallback.
For Electric Cloud, coordinators can be sharded deterministically by `source_id` and shape handle (for example 32 shards for a 200,000-reader source), with ephemeral state only and no Durable Object storage writes.
## Non-goals
- Sending shape rows or changelog bodies over WebSockets.
- Exposing a new public client API in the first release.
- Moving authentication into Electric.
- Multiplexing snapshots, SSE, refresh/catch-up requests, or requests without an existing handle and concrete offset.
Contributor guide
Research direction
Start by comparing the proposed /v1/shape/multiplex endpoint with the existing /v1/shape source selection, service-secret authentication, live timeout, cursor, and no-change behavior. Trace active-instance upgrades and watch lifecycle, then verify the protocol frames, eligibility rules, and retryable fallback semantics. Done means waiting state is multiplexed without sending shape data or changing ordinary HTTP responses.
Written by the indexing model from the issue text.
Assessment
- Domain
- api, backend-api-design, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100