jesseduffield / jesseduffield/horcrux

Suggestion: Use reed-solomon encoding to reduce data size

Open
#17 1 comment 9 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
5.1k
Forks
141
PR merge metrics
No merged PRs in 30d

Description

The current implementation of the program appears to copy the encrypted file into each horcrux, which becomes pretty storage intensive with a larger file and more horcruxes.

n is the number of horcruxes, t the number needed to ressurect, s is size of the file, t ≤ n
Reed-solomon encoding can split the original file into n pieces, each piece has a size of s/t. Any combination of t pieces can be used to recreate the original file.

So the total space used by the horcruxes would be n*(s/t) instead of n*s.

This should also ensure the integrity of the data as the reconstruction of the file should fail if a piece is modified.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the current implementation of encrypted-file copying into horcruxes and the reconstruction flow. Determine how Reed-Solomon encoding would affect fragment sizing, recovery, and modified-piece handling; done means the program stores smaller fragments and can reconstruct the original file from any t of n fragments while detecting invalid data.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.