filecoin-project / filecoin-project/filecoin-pin
feat: support batching adds to amortize addPieces fees
- Dominant language
- TypeScript
- Stars
- 26
- Forks
- 21
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 27
Description
`filecoin-pin add` commits one piece per invocation: `uploadToSynapse` calls `synapse.storage.upload()` once per CAR (`src/core/upload/synapse.ts:355`), which stores and commits in the same call. We should support batching adds, packing many items into fewer pieces and committing them in one `addPieces` transaction.
`addPieces` is charged per transaction, so adding a pile of small pieces one at a time can cost users a couple hundred dollars where the same content packed into one larger piece costs cents.
`add ` already covers content that is co-located, since it walks the tree into one CAR and one piece. The gap is items that arrive over time or from separate sources.
The migrate stack (#652 through #657) already does packing and batched commits for content fetched from IPFS gateways, so `pack-cars.ts` and `gc-window.ts` are the parts to build on.
Related: #651, #190.
Contributor guide
Assessment
This issue has not been assessed yet.