crossbario / crossbario/autobahn-python
Write coalescing
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 768
- PR merge metrics
- No merged PRs in 30d
Description
When sending many small messages (WebSocket and RawSocket), this can lead to significant syscall and CPU load.
Write coalescing buffers up writes to the underlying stream transport (TCP etc) in memory, and then only triggers actual writes every say 200ms.
Here is the idea: https://github.com/crossbario/autobahn-python/compare/master...oberstet:write_coalescing
txaio now has IBatchTimer, and we could reuse that!
Regarding the value of write coalescing, this is two-fold:
- it reduces the number of syscalls and CPU load (tradeoffs is increased latency vs throughput)
- it reduces the wire level traffic when run over TLS - we should make it such that writes to the underlying Twisted stream protocol only does 1 write
Regarding the latter: https://crossbario.com/blog/Dissecting-Websocket-Overhead/
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names WebSocket and RawSocket writes and suggests reusing txaio's IBatchTimer; it does not name files, tests, or entry points. Start by reviewing the write paths for both transports and the linked write_coalescing comparison. Done should include buffered writes, roughly 200ms triggering, and a single write to the underlying Twisted stream protocol.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100