hyperium / hyperium/hyper

Excessive per-connection memory cause by channel

Open
#4,057 5 comments 10 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-client B-upstream C-performance S-blocked
Dominant language
Rust
Stars
16.3k
Forks
1.8k
Avg merge
1d 22h
Merged PRs (30d)
14

Description

This is not a bug or anything, or even really behaving wrong, just an optimization potential.

Each HTTP/1.1 channel allocates an obvious 8k from INIT_BUFFER_SIZE: usize = 8192; this is fine and expected (although perhaps could use some configuration for "many connections with small payloads/less stringent performance requirements").

However, there is another ~8kb allocated as well per connection that I don't think is desirable. Each connection gets an unbounded dispatch channel. This in turn creates a channel that has 32 blocks (https://github.com/tokio-rs/tokio/blob/8f81e0814bd121683ebdfa21b494d1b77fb36403/tokio/src/sync/mpsc/mod.rs#L140 https://github.com/tokio-rs/tokio/blob/8f81e0814bd121683ebdfa21b494d1b77fb36403/tokio/src/sync/mpsc/block.rs#L47).

Each http request is ~250b, giving us another ~8kb for the channel here.

While in theory the SendRequest API does allow pushing a bunch of requests at once, practically speaking most are using this in front of a pooling client (hyper_util or otherwise) which is going to only send requests that are ready. So I would think we don't need for most cases unbounded, nor a 32 block buffer.

A theoretical nice option here, that allows the strange case of sending multiple requests but optimizes the common case, would be to just have the channel not use 32 blocks (which I imagine is for performance reasons not relevant to us?). However, tokio doesn't expose any such option.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the HTTP/1.1 channel allocation from INIT_BUFFER_SIZE and the per-connection dispatch channel exposed through SendRequest. Compare the memory cost of the current Tokio channel arrangement with the requirement to queue multiple requests, then verify that the chosen change reduces idle per-connection memory without breaking request dispatch behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.