Place a bound on the permitted version difference between TLogs in a recovery
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
Once upon a time, epochEnd used to enforce that the TLogs used in a recovery had to be within 5M versions / 5 seconds of eachother. This restriction was removed to support recovering an FDB cluster from disk snapshots, where the disk snapshots might have been taken more than 5s apart.
However, the current code specifies no restriction, which seems questionably excessive. This rendered clusters concerningly vulnerable to snapshot restore mistakes if processes from different cluster snapshots are mixed. I don't think there should be a reason to allow a recovery with TLogs more than 1hr apart, which would help to defend against a number of operator error cases with snapshots.
Contributor guide
Research direction
Start by locating epochEnd and the recovery code that validates TLogs, then trace how snapshot-based recovery supplies their versions and timestamps. Review existing recovery tests for the former 5M-version/5-second restriction. Done means the recovery path enforces an agreed maximum separation without breaking snapshots that were intentionally taken more than five seconds apart.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100