FilOzone / FilOzone/ipfs2foc

Investigate a hosted CAR aggregator service

Open
#69 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1
Forks
0
Avg merge
5d 21h
Merged PRs (30d)
1

Description

Aggregation has to happen on the migrator's own machine today. Content is downloaded, packed into an aggregate CAR, and uploaded, so every migrator pays local disk plus bandwidth in both directions.

The paths that avoid local disk all require a public HTTPS origin the provider can pull from: `redirect-serve` behind a tunnel, or `--source-relay`. Most consumers cannot provide one. CGNAT, corporate networks, and no-VPS users are simply excluded, and for the ipfs2filecoin campaign that is the largest single drop-off point.

PoDSI in Curio would remove the need for client-side aggregation, but we decided not to block on it and to aggregate client-side for now.

Worth investigating: a service that accepts a CID list, fetches from public gateways, builds aggregate CARs, and serves them at `/piece/{pieceCidV2}` for a provider to pull. That removes the local-disk requirement and the public-ingress requirement together.

Open questions:

- Who operates and funds it. The existing `packages/relay` Worker runs on a personal free-tier Cloudflare account and will not carry campaign traffic.
- Whether the trust model is acceptable. Bytes flow through a third party, though the provider still verifies CommP against the PieceCID the client committed, so a tampering relay fails the add rather than corrupting data.
- Rate limiting and abuse, given there is no registration step.

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing packages/relay Worker and how the migrator currently downloads, packs, and uploads aggregate CARs. Investigate the hosted service's operation, trust model, funding, rate limiting, and abuse controls. Done means a documented recommendation or design that resolves whether this service is viable, rather than an implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, cloud, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.