apple / apple/foundationdb

Use homomorphic hash to checksum Replicas

Open
#3,463 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Adrian (from Snowflake) [proposes](https://docs.google.com/document/d/1KXvlHncC9vunrXdW6kecwEcVEzL7uh07I6BBrdvOJ00/edit#) the use of a [homomorphic hash function open sourced by Facebook](https://engineering.fb.com/security/homomorphic-hashing/) to verify the consistency of partitions within FDB.

Currently we use the consistency checker to achieve this. However, in the case of a corruption it could potentially take the consistency checker a very long time to detect it. Furthermore this could also be used to verify the correctness of a backup and/or snapshot backup without comparing the whole database.

The above document discusses more details.

Contributor guide

Open the contributing guide

Research direction

Read the linked design document and Facebook's homomorphic-hashing article first, then trace the existing consistency checker and the backup and snapshot verification paths. Determine how the proposed hash would cover partition consistency and verification without comparing the whole database; done means the design and implementation are validated for those use cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
cryptography, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.