benoitc / benoitc/erlang_quic

Client holds a dead connection for the full idle timeout when the peer restarts: no PTO-based disconnect

Closed
#202 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Erlang
Stars
113
Forks
19
Avg merge
1d 22h
Merged PRs (30d)
35

Description

Follow-up from the same production evaluation as #201, this time on failure detection.

## Scenario

A client holds a long-lived connection to a server whose *process* restarts (the listener comes back on the same address within seconds). Captured with erlang_quic 1.7.0 as client against an msquic 2.4 server:

1. After the restart the client's packets hit a server with no connection state; it replies with stateless resets (observed as a stream of 30-byte short-header datagrams the client logs as `[short header decryption failed]`, one every few seconds, indefinitely).
2. The client cannot recognize them as resets: RFC 9000 section 10.3 recognition is exact-token, and msquic derives reset tokens from a per-process key, so the restarted process advertises tokens the surviving client never learned. This is not fixable on the client side by token matching, and any peer implementation with non-stable reset keys behaves the same after a restart.
3. Idle handling is correct (activity only bumps on successfully processed packets - verified), so the connection does die eventually. But with keep-alives configured, the natural `idle_timeout` is a multiple of the keep-alive interval: in our configuration 60 s of zombie connection during which every `send_data` is silently accepted into a black hole.

For comparison, msquic's client abandons the same dead path in ~16 s via its disconnect timeout: repeated PTO expiries with no acknowledgment are treated as path failure, independent of the idle timeout.

## Proposal: give up after persistent PTO expiry

RFC 9002 section 6.2 deliberately leaves this open ("endpoints could implement a maximum PTO count or time threshold"). Concretely:

- a `max_pto_backoff` (or `disconnect_timeout`) connection option: when ack-eliciting data has been outstanding continuously for that long - i.e. the PTO timer has fired and doubled N times with no packet successfully processed in between - close with a transport error to the owner (`{closed, pto_exhausted}` or similar);
- default it generously (say 30 s) or leave it opt-in to preserve current behavior;
- the state needed is already present (loss recovery tracks PTO count; the reset happens on any successfully processed packet).

This bounds the zombie window for *every* undetectable-death case (process restart, NAT box reboot, silent path loss with an active sender), not just the stateless-reset one, and matches what deployed stacks do.

The heuristic alternative - treating N consecutive short-header decryption failures from the peer's address as probable reset - would also work for this specific case, but PTO exhaustion is the more principled and general signal, and cannot be triggered by an off-path attacker injecting garbage.

Reproduction: any request/response or streaming workload where the server process restarts mid-stream while the client has data outstanding; the client stalls until idle timeout instead of failing fast. Happy to validate a patch against our integration suite where we caught this.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the loss-recovery PTO count and timer handling described in the issue, then inspect the integration suite mentioned for the restart scenario. Define how the connection option and owner notification should behave, and verify that persistent PTO expiry closes the connection while successfully processed packets reset the failure state.

Written by the indexing model from the issue text.

Assessment

Tech stack
erlang
Domain
networking
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.