Excessive per-connection memory cause by channel
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 16.3k
- Forks
- 1.8k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 14
Description
This is not a bug or anything, or even really behaving wrong, just an optimization potential.
Each HTTP/1.1 channel allocates an obvious 8k from INIT_BUFFER_SIZE: usize = 8192; this is fine and expected (although perhaps could use some configuration for "many connections with small payloads/less stringent performance requirements").
However, there is another ~8kb allocated as well per connection that I don't think is desirable. Each connection gets an unbounded dispatch channel. This in turn creates a channel that has 32 blocks (https://github.com/tokio-rs/tokio/blob/8f81e0814bd121683ebdfa21b494d1b77fb36403/tokio/src/sync/mpsc/mod.rs#L140 https://github.com/tokio-rs/tokio/blob/8f81e0814bd121683ebdfa21b494d1b77fb36403/tokio/src/sync/mpsc/block.rs#L47).
Each http request is ~250b, giving us another ~8kb for the channel here.
While in theory the SendRequest API does allow pushing a bunch of requests at once, practically speaking most are using this in front of a pooling client (hyper_util or otherwise) which is going to only send requests that are ready. So I would think we don't need for most cases unbounded, nor a 32 block buffer.
A theoretical nice option here, that allows the strange case of sending multiple requests but optimizes the common case, would be to just have the channel not use 32 blocks (which I imagine is for performance reasons not relevant to us?). However, tokio doesn't expose any such option.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the HTTP/1.1 channel allocation from INIT_BUFFER_SIZE and the per-connection dispatch channel exposed through SendRequest. Compare the memory cost of the current Tokio channel arrangement with the requirement to queue multiple requests, then verify that the chosen change reduces idle per-connection memory without breaking request dispatch behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100