Small sends do not coalesce: one datagram per send_data, 10x throughput left on the table
- Dominant language
- Erlang
- Stars
- 113
- Forks
- 19
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 35
Description
We are evaluating erlang_quic as a production transport for a pub/sub system whose dominant traffic is many small messages multiplexed over one connection (hundreds of streams, messages of 50-500 bytes, bursts of tens of thousands). On that shape the send path currently pays one datagram (or more) per `send_data`, and it is the single largest performance gap we measured against our current C-based stack.
## Measurement
Loopback, one connection, one stream, 20 000 messages of 100 bytes, 1.7.0. UDP sends counted with call tracing on the client connection process only:
| mode | rate | `gen_udp:send` per message |
|---|---|---|
| `send_data/4` (sync) per message | 15.5k msg/s | 2.00 |
| `send_data_async/4` per message | 21.4k msg/s | 1.49 |
| 10 messages concatenated per `send_data` | **154k msg/s** | 0.20 |
The 10x jump from application-side batching is the tell: the cost is per-send packetization, not bytes. Every `send_data` becomes its own STREAM frame in its own packet in its own datagram, so a 100-byte message ships with ~60 bytes of packet overhead plus a syscall, and the ACK-eliciting packet rate explodes (the 2.00 includes ACK traffic provoked at the peer).
For comparison, msquic buffers app sends and fills packets to the MTU: the same message stream coalesces into ~10-14 messages per datagram.
## Proposal: fill packets from queued small frames at packetization time
The machinery is nearly there already. `dequeue_small_stream_frame_tuple/1` opportunistically piggybacks one queued small stream frame onto an outgoing ACK packet. Generalizing that:
1. When building a 1-RTT packet for a stream frame, after placing the triggering frame, keep draining further queued small stream frames (any stream, respecting per-stream and connection flow control and cc) into the same packet until the datagram budget is spent - the classic packet-fill loop.
2. Route small direct sends through the send queue instead of packetizing immediately when the queue is non-empty, so consecutive `send_data_async` casts that are already sitting in the connection process mailbox get the chance to share datagrams. A pure mailbox-drain approach (process all pending `{send_data,...}` casts before packetizing) would achieve the same without any timer.
3. No aggregation delay needed: only coalesce what is already there. Latency for a lone small send stays one packetization, unchanged.
Point 2 is what turns the 2.00 into something near msquic's behavior for bursty senders: under load the mailbox always has more sends waiting, and each scheduler pass flushes them as full datagrams; when idle, nothing changes.
We measured the equivalent effect from the application side (the 154k row) but doing it in the library benefits every caller and keeps message boundaries out of application code.
Happy to test any branch against our workload (same setup as the numbers above, plus our integration suite), or to take a stab at the patch if you would rather review than write it.
Contributor guide
No contributing guide indexed for this repository
Research direction
Locate send_data/4, send_data_async/4, and dequeue_small_stream_frame_tuple/1 in the send and packetization paths, then trace how queued stream frames become datagrams. Compare the existing ACK piggyback behavior with 1-RTT packet construction and mailbox handling; done means queued small frames coalesce within packet and flow-control limits without delaying lone sends, with throughput and UDP-send counts verified against the reported workload.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- erlang
- Domain
- networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100