python-trio / python-trio/trio
Tune default receive buffer size
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.3k
- Forks
- 431
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 6
Description
Since #1123, our streams have a default receive size baked in. For example, the initial default for SocketStream is 64KiB. Is this the best size? We have no idea. It would be nice to have some idea.
On #1123, @oremanj raised some of the issues that might affect this:
One wrinkle: AFAIK, each call to
socket.recv()allocates a newbytesobject that is large enough for the entire given chunksize. If large allocations are more expensive, passing a too-large buffer is probably bad for performance. (The allocators I know of use 128KB as their threshold for "this is big, mmap it instead of finding a free chunk" but if one used 64KB instead and we got a mmap/munmap pair on each receive, that feels maybe bad?)
My intuition favors a much lower buffer size, like 4KB or 8KB, but I also do most of my work on systems that are rarely backlogged, so my intuition might well be off when it comes to a high-throughput Trio application.
Another option we could consider: the socket owns a receive buffer (bytearray) which it reuses, calls
recv_into(), and extracts just the amount actually received into abytesfor returning. Downside: spends 64KB (or whatever) per socket in steady state. Counterpoint: the OS-level socket buffers are probably much larger than that (but I don't know how much memory they occupy when the socket isn't backlogged).
It's true that if you do sock.recv(N), Python has to malloc an N byte buffer, and then realloc back down to the actual size, so there is some cost to using a large N. The consequences of that aren't very obvious to me though. Most allocators have countermeasures against repeatedly growing/shrinking the heap like that (e.g. search malloc hysteresis). If doing our own buffer management is worthwhile, then note that we could potentially share a buffer between all sockets in the same thread. But of course the real answer to all of this is that we have to measure.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the default receive size for SocketStream and the socket.recv() allocation behavior described in the issue. Measure the performance and memory effects of candidate buffer sizes, including the possible recv_into() buffer-management approach. Done means selecting and implementing a data-supported default or buffer strategy, with measurements documenting the result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100