libp2p / libp2p/go-libp2p-pubsub
Pubsub should consolidate its peering
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 364
- Forks
- 221
- PR merge metrics
- No merged PRs in 30d
Description
I've been playing with GossipSub recently, and I noticed that pubsub doesn't consolidate its peering by itself. To be clear, I'm not talking about the general peer discovery (as in, "find peer for that topic as I have none"), I'm talking about pubsub receiving messages from peers in that topic, but not trying to connect directly even though the gossip topology is below the requirements.
As I understand, the expected solution is to pair GossipSub with a DHT and the WithDiscovery() option, so that pubsub can ask for more peer when below the topology requirements, but that's a quite heavy solution in my opinion, especially when pubsub **already know those peers**, just not their multiaddr.
If no DHT is setup, the peering is very brittle. If there is a single bootstrap node, every peer's communication will go through that bootstrap in a star topology, with an obvious single point of failure.
What I've ended up doing is adding two extra messages in my tiny app protocol, and package that into a WithDiscovery():
1. DISC_QUERY: broadcast an ask for participants to reply with their addrInfo
2. DISC_RESP: reply with an AddrInfo
It works really well, the topology grows and is resilient. When some peers are in the same LAN, they even find each other directly without mDNS. .... but that feels gross and really sub-optimal. Flooding the topic with answers is bad, having an external component is bad, having to integrate that into the app protocol is bad.
Would it be possible to have an opt-in solution so that pubsub itself query for other peers, as part of the pubsub protocol? That would be so much cleaner and efficient.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing GossipSub peering behavior and the WithDiscovery() option described in the issue. Determine how pubsub could discover known peers without a DHT, then define an opt-in protocol design and tests that demonstrate topology consolidation without flooding or an external app protocol.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100