Performance and correctness series (#203-#219): summary, results, and merge options
- Dominant language
- Erlang
- Stars
- 113
- Forks
- 19
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 35
Description
Hi Benoit,
Over the last week we have been running erlang_quic as the transport for our production event-streaming system (as the client on ~50k edge devices, and evaluating it server side), and we did a deep performance and correctness pass. The result is the PR series #203 through #219 plus a few follow-ups still on our fork. I wanted to give you the full picture in one place, since 17 individual PRs without context is a lot to digest, and to offer some options for how to merge it with the least effort on your side.
## Results
Single stream bulk transfer, 200 MiB over loopback, client and server in separate VMs, median of 6 runs, measured with a bounded-async sender:
| build | socket backend | gen_udp |
|---|---|---|
| 1.8.1 (start) | ~10 MiB/s, unstable | ~40 MiB/s, erratic |
| with the full series | ~805 MiB/s | ~530 MiB/s |
For reference on the same machine: Erlang TLS (ssl:send) does ~480 MiB/s and msquic-based quicer ~840 MiB/s. So erlang_quic ends up faster than Erlang distribution's own TLS and within a few percent of msquic, while staying pure Erlang (the crypto NIF in #216 is optional with a byte-exact fallback). Idle connection memory also dropped from ~180 KiB to ~8 KiB per connection (#207), which matters at our device counts.
All of it is exercised daily by our integration suite (connection restarts, failovers, bulk catch-up, NAT rebinding) and the full eunit/CT suites are green on every PR.
## What the series contains
Correctness fixes, several of them RFC-conformance issues that cause real stalls or data loss:
- #204 GRO cmsg size (silent truncation of receive trains)
- #205 async-send data loss and a send-queue ordering deadlock
- #206 overlapping stream frames (RFC 9000 section 2.2 repacketization; permanent receive stall)
- #209 MAX_DATA update trigger (window cliff after the first 50 MB)
- #210 loss-time threshold per RFC 9002 section 6.1.2
- #211 ACK-range cap per RFC 9000 section 13.2.4 (unbounded CPU under loss)
- #212 loss retransmissions subject to cwnd
- #218 switching away from a retired DCID (RFC 9000 section 5.1.2; permanent deafness after peer restart)
Performance:
- #203 small-send coalescing (your #201)
- #213 configurable ACK decimation
- #214 per-drain burst budget
- #215 GRO truncation fix in the socket backend
- #217 GSO uniform-run batching
- #219 pacing burst allowance scaled with the pacing rate (this one is independent of everything else and applies directly to main; it alone roughly doubles top-end throughput on fast links)
Optional:
- #207 idle-connection hibernation
- #208 disconnect timeout (PTO-style give-up on unresponsive peers)
- #216 AEAD context NIF (opt-in accelerator, pure-Erlang fallback unchanged)
On top of the series, our fork carries follow-ups that depend on it and that we can PR once the base lands: fused packet-protect/open NIF paths, an ordered (gb_trees) reassembly buffer, lean receive fast paths, and batched CC/loss bookkeeping on the send path. Those account for roughly the last 2x.
## Merge options, your pick
1. Review the series individually. Suggested order: #219 first (standalone), then #203, #206, #216 (the three roots), then the rest, which apply in numeric order.
2. One rollup PR with the whole series (and optionally the follow-ups), and we close the individual PRs. Our fork branch is effectively this rollup already and is what we run our test matrix against, so it is cheap for us to produce and it gives you one green CI run to judge.
3. A hybrid: rollup for the mechanical fixes and performance work, individual review for the judgment calls (#207, #208, #216, #219).
Happy to go whichever way suits you, to split or squash anything, and to add tests where you want more coverage. And of course to answer questions on any individual change.
Thanks for erlang_quic!
Johan
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with PR #219, then review the three stated roots: #203, #206, and #216, followed by the remaining PRs in numeric order. Compare the full series with the fork branch and its green eunit/CT and integration-suite results. Done means selecting and carrying out a merge strategy for the series, with any requested tests or review changes addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- erlang
- Domain
- networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100