jesseduffield / jesseduffield/horcrux
Suggestion: Use reed-solomon encoding to reduce data size
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 141
- PR merge metrics
- No merged PRs in 30d
Description
The current implementation of the program appears to copy the encrypted file into each horcrux, which becomes pretty storage intensive with a larger file and more horcruxes.
n is the number of horcruxes, t the number needed to ressurect, s is size of the file, t ≤ n
Reed-solomon encoding can split the original file into n pieces, each piece has a size of s/t. Any combination of t pieces can be used to recreate the original file.
So the total space used by the horcruxes would be n*(s/t) instead of n*s.
This should also ensure the integrity of the data as the reconstruction of the file should fail if a piece is modified.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the current implementation of encrypted-file copying into horcruxes and the reconstruction flow. Determine how Reed-Solomon encoding would affect fragment sizing, recovery, and modified-piece handling; done means the program stores smaller fragments and can reconstruct the original file from any t of n fragments while detecting invalid data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100