oasisprotocol / oasisprotocol/oasis-core

Erasure coding

Open
#504 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

c:storage p:3
Dominant language
Go
Stars
369
Forks
151
Avg merge
1d 3h
Merged PRs (30d)
6

Description

As mentioned in #501 , we need to enable data fetch between the compute committees in two consecutive epochs. What if we have some packet loss, or some of the nodes to fetch data from go down.

The easiest way to fix this fault-tolerance issue is by using Forward Error Correction (FEC) coding scheme in wireless communication domain. We can directly use existing FEC coding schemes like erasure coding, spinal codes, etc.

The basic idea of FEC coding is to add some global redundancy on each data packet. Given the condition we may have some packets loss or data sender goes down, if we can successfully receive any N packets from any nodes, we can still recover the whole dataset (say with size of K packets), where N > K but just 1.1-1.3x the size of K.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the requirements referenced in issue #501, then clarify how compute committees exchange data across consecutive epochs and how packet loss or unavailable nodes should be handled. Done should mean that the dataset can be recovered from any N received packets, with N only modestly larger than K, but no implementation files or tests are identified here.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
blockchain, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.