br: blocklist files written by older BR always fail checksum after #64465, breaking log truncate and repeat PITR
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
**Affected branches**: master (unfixed as of 66be7a615b) and release-8.5 (≥ v8.5.6, via cherry-pick #65018). Introduced by #64465.
### What happened
`LogRestoreTableIDsBlocklistFile` (`br/pkg/restore/misc.go`) was changed incompatibly: proto field 2 (`snapshot_backup_ts`) was dropped and replaced by field 7 (`restore_start_ts`) — but the field is covered by the file checksum.
For any blocklist file written by an older BR:
1. field 2 is discarded as unknown on unmarshal, field 7 is absent → `RestoreStartTs = 0`;
2. the checksum is recomputed with 0 instead of the real `snapshot_backup_ts` → **always mismatches**;
3. `unmarshalLogRestoreTableIDsBlocklistFile` returns a hard `checksum mismatch ... may be corrupted` error, and `fastWalkLogRestoreTableIDsBlocklistFile` aborts the whole walk.
Two affected paths:
- **`br log truncate`** (`TruncateLogRestoreTableIDsBlocklistFiles`): any log-backup storage that ever had a PITR done by an older BR can no longer be truncated once `--until` passes the old file's commit TS. This is a routine retention operation — it breaks silently after a BR upgrade.
- **Repeat PITR pre-check** (`CheckTableTrackerContainsTableIDsFromBlocklistFiles`): restoring with `restored-ts >= old snapshotBackupTs` and `start-ts < old restoreCommitTs` reads the old file and fails.
### Reproduction (verified end-to-end)
Constructed a valid old-format blocklist file (fields 1/2/6/3/5 + legacy checksum) in a local storage and ran a current build of BR:
```
$ br log truncate --storage local:///tmp/s --until 400000000000000200 -y
Error: checksum mismatch (calculated checksum is w5qjCoQZFzVBX27Fwy9lm3FFOKIt8l7k+2NNbOrswbw= but the recorded checksum is rDbIQoFWoTwVKhDZIGAjPduBYwdD0otZBiQ66ybrrT0=), the log restore table IDs blocklist file may be corrupted
```
Cross-verified: the `recorded` value equals the legacy algorithm over the real `snapshot_backup_ts`, and the `calculated` value equals the new algorithm with `RestoreStartTs=0` — matching the error text character-for-character. The same file in the new format (field 7) truncates successfully.
### Suggested fix
Tolerate legacy files instead of hard-failing, e.g. fall back to the filename-parsed `(restoreCommitTs, S)` values with a warning (for truncate, deleting a file should not require trusting its contents), or keep proto field 2 reserved and fall back to the legacy checksum when `RestoreStartTs == 0`.
Contributor guide
Research direction
Start in br/pkg/restore/misc.go, tracing unmarshalLogRestoreTableIDsBlocklistFile, fastWalkLogRestoreTableIDsBlocklistFile, and the truncate and repeat-PITR callers. Reproduce with the documented br log truncate command using a legacy blocklist file, then verify legacy files no longer abort checksum validation while current-format files still work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100