define a new stream multiplexer
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 1.8k
- Forks
- 320
- Avg merge
- 11d 15h
- Merged PRs (30d)
- 1
Description
yamux is bad, and mplex is even worse. We need a better stream muxer for TCP connections.
Specifically, we need a muxer that has:
- back pressure on the stream level, i.e. a limit how much outstanding data can be sent on a single stream
- back pressure on the connection level, i.e. a limit how much outstanding data can be sent on the entire connection (typically, this limit will be significantly lower than the number of streams multiplied by the stream data limit)
- back pressure for new streams, i.e. a limit how many streams can be active at any given point
Ideally, all these limits can be set dynamically during the lifetime of a connection (e.g. allowing with small limits, and increasing them later).
The QUIC stream multiplexer fulfills all these properties, and we know that it works well. We could adapt it to work over TCP.
I've started writing up how this could look like: https://hackmd.io/JXgEHrA5SGy7ioKCp-OnGQ. This document will need a lot more love before we can implement something.
We will also need to coordinate the implementation work across languages: This effort only makes sense if we have implementations in Go, Rust and JavaScript. Once we have those, we can start phasing out old stream multiplexers, for example by reducing their limits (so performance decreases, but we don't break connectivity to legacy nodes).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the design document at https://hackmd.io/JXgEHrA5SGy7ioKCp-OnGQ and compare its proposal with the QUIC stream multiplexer requirements described here. Done means completing the specification and coordinating compatible implementations in Go, Rust, and JavaScript, including stream, connection, and new-stream back-pressure limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, javascript, rust
- Domain
- networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100