ipfs-shipyard / ipfs-shipyard/integration-mini-projects
Represent hypercore in IPLD
- Dominant language
- No language data
- Stars
- 14
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
6 months ago, I did some initial experimentation to see if I could model the [hypercore](https://github.com/mafintosh/hypercore) data structure in IPLD. Hypercore is the append-only log library used by Dat.
https://github.com/jimpick/hypercore-simple-ipld
As hypercore data is stored in [simple binary tree](https://datprotocol.github.io/how-dat-works/), it is very easy to model with IPLD. I didn't get as far as storing signatures, or the bitfield used by the replication protocol, but those shouldn't be hard to model.
I was experimenting with implementing the [random-access-storage](https://github.com/random-access-storage/random-access-storage) interface, so, if completed, it would be possible to read/write hypercore data from an IPFS node directly (similar to how hypercore can use disk storage, or memory). It would be interesting to see what the performance profile is like ... I suspect it should be similar to MFS.
If the storage interface is implemented, it should also be possible to run [hyperdrive](https://github.com/mafintosh/hyperdrive), [hypertrie](https://github.com/mafintosh/hypertrie) and [hyperdb](https://github.com/mafintosh/hyperdb) on top of that. [hypercored](https://github.com/mafintosh/hypercored) could write directly to IPFS or ipfs-cluster.
Because IPFS will deduplicate blocks, this may be a cost saving to some people who are trying to store large amounts of data using Dat. Additionally, ipfs-cluster may be a nice way to manage large amounts of Dat data. And Filecoin could be used to keep archives alive for long periods of time.
It would also be interesting to try to run hypercore replication over libp2p, and to try different approaches for content discovery, but that would be a separate project.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.