libp2p / libp2p/specs

define a new stream multiplexer

Open
#377 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
1.8k
Forks
320
Avg merge
11d 15h
Merged PRs (30d)
1

Description

yamux is bad, and mplex is even worse. We need a better stream muxer for TCP connections.

Specifically, we need a muxer that has:

  • back pressure on the stream level, i.e. a limit how much outstanding data can be sent on a single stream
  • back pressure on the connection level, i.e. a limit how much outstanding data can be sent on the entire connection (typically, this limit will be significantly lower than the number of streams multiplied by the stream data limit)
  • back pressure for new streams, i.e. a limit how many streams can be active at any given point

Ideally, all these limits can be set dynamically during the lifetime of a connection (e.g. allowing with small limits, and increasing them later).

The QUIC stream multiplexer fulfills all these properties, and we know that it works well. We could adapt it to work over TCP.

I've started writing up how this could look like: https://hackmd.io/JXgEHrA5SGy7ioKCp-OnGQ. This document will need a lot more love before we can implement something.
We will also need to coordinate the implementation work across languages: This effort only makes sense if we have implementations in Go, Rust and JavaScript. Once we have those, we can start phasing out old stream multiplexers, for example by reducing their limits (so performance decreases, but we don't break connectivity to legacy nodes).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the design document at https://hackmd.io/JXgEHrA5SGy7ioKCp-OnGQ and compare its proposal with the QUIC stream multiplexer requirements described here. Done means completing the specification and coordinating compatible implementations in Go, Rust, and JavaScript, including stream, connection, and new-stream back-pressure limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, javascript, rust
Domain
networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.