python-trio / python-trio/trio

Tune default receive buffer size

Open
#1,139 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

performance
Dominant language
Python
Stars
7.3k
Forks
431
Avg merge
2d 17h
Merged PRs (30d)
6

Description

Since #1123, our streams have a default receive size baked in. For example, the initial default for SocketStream is 64KiB. Is this the best size? We have no idea. It would be nice to have some idea.

On #1123, @oremanj raised some of the issues that might affect this:

One wrinkle: AFAIK, each call to socket.recv() allocates a new bytes object that is large enough for the entire given chunksize. If large allocations are more expensive, passing a too-large buffer is probably bad for performance. (The allocators I know of use 128KB as their threshold for "this is big, mmap it instead of finding a free chunk" but if one used 64KB instead and we got a mmap/munmap pair on each receive, that feels maybe bad?)

My intuition favors a much lower buffer size, like 4KB or 8KB, but I also do most of my work on systems that are rarely backlogged, so my intuition might well be off when it comes to a high-throughput Trio application.

Another option we could consider: the socket owns a receive buffer (bytearray) which it reuses, calls recv_into(), and extracts just the amount actually received into a bytes for returning. Downside: spends 64KB (or whatever) per socket in steady state. Counterpoint: the OS-level socket buffers are probably much larger than that (but I don't know how much memory they occupy when the socket isn't backlogged).

It's true that if you do sock.recv(N), Python has to malloc an N byte buffer, and then realloc back down to the actual size, so there is some cost to using a large N. The consequences of that aren't very obvious to me though. Most allocators have countermeasures against repeatedly growing/shrinking the heap like that (e.g. search malloc hysteresis). If doing our own buffer management is worthwhile, then note that we could potentially share a buffer between all sockets in the same thread. But of course the real answer to all of this is that we have to measure.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the default receive size for SocketStream and the socket.recv() allocation behavior described in the issue. Measure the performance and memory effects of candidate buffer sizes, including the possible recv_into() buffer-management approach. Done means selecting and implementing a data-supported default or buffer strategy, with measurements documenting the result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.