ProvideMany: high memory usage when providing tens of millions of CIDs
Open
need/triage
- Dominant language
- Go
- Stars
- 316
- Forks
- 163
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 8
Description
When using BatchProviding, we are not really batching, but sending all the CIDs at the same time to the Router implementing ProvideMany.
To avoid collateral problems, we should actually batch the calls to ProvideMany.
This will help with Reframe Router implementation (https://github.com/ipfs/go-delegated-routing) to avoid huge JSON payloads sent to the server.
We need to find good defaults to still keep FullRT DHT implementation with good performance numbers.
That is like 1/10th of the memory spike observed, we are still searching for other possible problems.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.