apple / apple/foundationdb

Place a bound on the permitted version difference between TLogs in a recovery

Open
#6,090 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Once upon a time, epochEnd used to enforce that the TLogs used in a recovery had to be within 5M versions / 5 seconds of eachother. This restriction was removed to support recovering an FDB cluster from disk snapshots, where the disk snapshots might have been taken more than 5s apart.

However, the current code specifies no restriction, which seems questionably excessive. This rendered clusters concerningly vulnerable to snapshot restore mistakes if processes from different cluster snapshots are mixed. I don't think there should be a reason to allow a recovery with TLogs more than 1hr apart, which would help to defend against a number of operator error cases with snapshots.

Contributor guide

Open the contributing guide

Research direction

Start by locating epochEnd and the recovery code that validates TLogs, then trace how snapshot-based recovery supplies their versions and timestamps. Review existing recovery tests for the former 5M-version/5-second restriction. Done means the recovery path enforces an agreed maximum separation without breaking snapshots that were intentionally taken more than five seconds apart.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.