[ipfs/go-bitswap] Reduce Broadcasting
- Dominant language
- Go
- Stars
- 316
- Forks
- 163
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 8
Description
Currently, when we can't find some content, we add it to the "broadcast" list and ask every connected peer. Unfortunately, this means that bitswap nodes under heavy load will:
1. Take a while to find blocks (heavy load).
2. Believe these blocks "can't be found" and broadcast requests.
3. Slow down even more.
Furthermore, these massive broadcast lists end creating a lot of unnecessary traffic.
Insight: Sessions tend to be highly correlated (by design) so asking for more than one CID in a session is unlikely to give us any extra information. At the very least, broadcasting a request for a new CID B won't get us any closer to finding some previously broadcast CID A.
Given this insight, we can trim down the broadcast list to one want per _session_, which is much more manageable. It also doesn't _really_ matter _which_ want, as long as it's relevant.
Proposal:
1. When a session has no peers and/or is "stuck", broadcast a single want request. This want should probably be the oldest still outstanding want in the session.
2. When broadcasting, rank peers by "class" and iteratively widen the broadcast. Some heavily loaded nodes like public gateways may simply ignore some classes and/or "batch" requests.
1. Peers who previously gave us blocks but got kicked out of the session (IIRC, this class exists but I'm not sure).
2. Peers we've never seen before (never sent a broadcast to).
3. Peers we've previously sent broadcasts to.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.