Support for rabin chunking files
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 273
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
This project is awesome! I've always wanted to implement a userspace content addressable filesystem and one fine day I found this which does it in just a few lines of js code : )
Instead of content addressing at the file level, how about splitting files into chunks using rabin fingerprints and content addressing those. That would improve deduplication, make snapshotting more space-efficient and also speed up synchronization.
Thoughts about this? Would it be worth the overhead?
I'm interested in implementing this functionality.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files or tests. Start by reading how the filesystem currently uses fuse, hyperlog, leveldb, and node to address whole files, then determine the chunking and synchronization scope. Done would require an agreed design and implementation plan for Rabin-based chunk addressing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- operating-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100