IPFS support
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 24
- Forks
- 16
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 1
Description
## General considerations
- We are not willing to run our own IPFS nodes at this point, therefore, we need to upload (pin) the files to cloud provider, like [Pinata],(https://www.pinata.cloud/) through their proprietary API. To ensure the cloud provider didn't tamper with the contents of the file, we probably want to add an extra step of calculating the content hash locally and comparing it with the address, returned after upload.
- Files could be accessed by HTTP through [public](https://ipfs.github.io/public-gateway-checker/) or [private](https://docs.pinata.cloud/gateways/dedicated-gateways) gateway providers. To operate in a trustless manner, the content hash should be calculated and matched against the returned CID. The minimal unit, which can be validated, is an entire file. An [example](https://ipfs.io/ipfs/Qme7ss3ARVgxv6rXqVPiikMJ8u2NLgmgszg13pYrDKEoiu) of IPFS URL, which uses public gateway.
- With content-based addressing, the target URL is not input, but output parameter, because it includes content hash. It may require a slightly different flow, compared to S3. It also may present several challenges with playlist generation e.g. a segment should be fully uploaded, before playlist entry can be added; playlist URL will change every time it's updated. [IPNS](https://docs.ipfs.tech/concepts/ipns/) and [DNSLink](https://docs.ipfs.tech/concepts/dnslink/) are mechanisms which could be used to address that.
## VOD
### Input
- IPFS URLs do not explicitly contain file type. Most widely used IPFS gateway implementation automatically sets `Content-type` header in HTTP GET response, we can use that, but it's [not a mandatory requirement for gateway implementations](https://github.com/ipfs/specs/blob/main/http-gateways/PATH_GATEWAY.md#content-type-response-header).
### Output
- `ipfs://` URL support should be added to [go-tools](https://github.com/livepeer/go-tools/issues/3) and `catalyst-uploader`. It will use Pinata API to pin the file and return Pinata IPFS gateway URL. Content hash matching should be implemented as well.
- when segment is uploaded, Mist needs to use URL returned from `catalyst-uploader` in the playlist
- TBD: what should be the playlist behavior? It seems that IPNS is not available on Pinata. It seem to support [directory wrapping](https://flyingzumwalt.gitbooks.io/decentralized-web-primer/content/files-on-ipfs/lessons/wrap-directories-around-content.html), with pretty obscure API, which may allow to address the file by name.
## Live
- The main issue is dynamic playlist files
- TBD
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the referenced go-tools issue and the catalyst-uploader entry point, then trace how Mist currently handles S3 uploads and playlist updates. The Pinata API, local content-hash validation, returned gateway URLs, and live playlist behavior all need decisions; done is not defined until the VOD and Live flows are specified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cloud, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100