Guarantee minimal dataloss in DR
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
With the current DR solution (and also with the new backup strategy IIUC) if one loses a whole cluster, the amount of data loss would typically be in the order of seconds.
However, I don't think there are any hard guarantees. So if the backup-agents fail, or if one is unable to write to the blob-storage for an extended period of time, data loss could be much worse. This is due to the asynchronous nature of DR.
One possible solution is to stop accepting commits if the DR lag is too high. This probably should be something that can be configures by the user.
Contributor guide
Research direction
Start by reviewing the current DR solution and the new backup strategy described in the issue, focusing on how backup-agent failures and blob-storage outages affect DR lag. Define what configurable lag threshold and commit behavior would provide a hard data-loss guarantee, then identify the relevant implementation and test entry points; done means the guarantee and configuration are documented and covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100