libp2p / libp2p/go-libp2p-pubsub

Handling large number of topic subscriptions

Open
#402 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

  • #354 by @requilence — closed without merging
exp/expert help wanted P2
Dominant language
Go
Stars
364
Forks
221
PR merge metrics
No merged PRs in 30d

Description

Currently if a node subscribes to enough topics such that the RPC that is used to send topic subscription announcements exceeds the maximum message size (defaults to 1MiB) then we'll silently fail and stop announcing any of our topics to newly connected peers.

We should more gracefully error, or inform the user, when they've exceeded the maximum number of topics they can subscribe to and additionally it'd be nice to have a mechanism in place to allow for a larger number of subscriptions.

- [ ] (Beginner) Error when exceeding maximum topic limit
- Easiest: Just check if the (hello packet) message size is too large and log an error
- More correct: When subscribing to a new topic check if the new topic would force the hello packet to exceed the maximum size and if so return an error instead of allowing the topic to be subscribed to.
- [ ] (Expert) Allow sending more topics than fit in a single message

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the hello packet and the RPC used for topic subscription announcements, then trace how message-size failures are handled when many topics are subscribed. The beginner scope is complete when exceeding the maximum is reported or rejected instead of silently stopping announcements; the separate multi-message design is explicitly marked expert work.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.