electric-sql / electric-sql/electric
Elixir client ignores Retry-After on 503 responses
- Dominant language
- TypeScript
- Stars
- 10.4k
- Forks
- 375
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 18
Description
## Problem
The Elixir client appears to ignore `Retry-After` headers on retryable HTTP responses, including `503 Service Unavailable`.
This matters for server-side load shedding/admission control: if the sync service returns `503` with `Retry-After`, Elixir clients will not honor the server-selected retry delay. Instead they keep retrying using their own jittered exponential backoff schedule, capped at 32 seconds.
## Current behavior
`Electric.Client.Fetch.HTTP` installs a custom Req retry callback:
- `packages/elixir-client/lib/electric/client/fetch/http.ex`
- `build_request/2` passes `retry: &retry(&1, &2, retry_delay_fun, is_transient_fun, timeout)`
- `retry/5` computes `delay_ms = request_delay(request, retry_delay_fun)`
- the callback returns `{:delay, delay_ms}`
The default retry delay is:
```elixir
defp retry_delay(n) do
# Full jitter strategy (AWS recommended), minimum 1ms:
# random_between(1, min(cap, base * 2^n))
delay = min(32_000, Integer.pow(2, n) * 1000)
:rand.uniform(delay)
end
```
So the Elixir client uses client-side full jitter over roughly:
```text
1s -> 2s -> 4s -> 8s -> 16s -> 32s capped
```
Req itself has built-in `Retry-After` handling for `429` and `503`, but that path is bypassed when a custom retry callback returns `{:delay, milliseconds}`. In this client, there is no local parsing of `Retry-After`, and searching `packages/elixir-client` shows no use of `retry-after` / `Retry-After`.
## Expected behavior
For `503` responses with `Retry-After`, the Elixir client should honor the server-provided delay. It should probably match the TypeScript client semantics:
```text
wait = max(server Retry-After, client jittered backoff)
```
That preserves client-side backoff when the server does not provide guidance, while allowing server-side admission control to spread retries during overload.
## Why this matters
The sync service can use `503 + Retry-After` to reduce reconnect storms and avoid retry amplification during startup/restart/load-shed scenarios. TypeScript clients currently treat `Retry-After` as one candidate delay and use whichever is higher. Elixir clients do not, so they may continue retrying earlier than the server requested.
This is especially problematic if the server sends larger `Retry-After` values under pressure: TypeScript clients will wait at least that long, while Elixir clients remain capped by their own 32s jitter window.
## Notes
Retryable statuses in the Elixir client are currently `408, 429, 500, 502, 503, 504`, plus selected transport errors. `409` is handled separately by `Poll` as `must_refetch`, so this issue is specifically about retryable transient HTTP responses such as `503`.
Contributor guide
Assessment
This issue has not been assessed yet.