apple / apple/foundationdb

Guarantee minimal dataloss in DR

Open
#3,631 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

With the current DR solution (and also with the new backup strategy IIUC) if one loses a whole cluster, the amount of data loss would typically be in the order of seconds.

However, I don't think there are any hard guarantees. So if the backup-agents fail, or if one is unable to write to the blob-storage for an extended period of time, data loss could be much worse. This is due to the asynchronous nature of DR.

One possible solution is to stop accepting commits if the DR lag is too high. This probably should be something that can be configures by the user.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the current DR solution and the new backup strategy described in the issue, focusing on how backup-agent failures and blob-storage outages affect DR lag. Define what configurable lag threshold and commit behavior would provide a hard data-loss guarantee, then identify the relevant implementation and test entry points; done means the guarantee and configuration are documented and covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.