cockroachdb / cockroachdb/cockroach

backup: memory monitored restore may not yield the latest key AOST from a revision history backup

Open
#107,378 1 comment 0 reactions 0 assignees View on GitHub
A-disaster-recovery C-bug T-disaster-recovery
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

When the restore data processor[ preempts](https://github.com/cockroachdb/cockroach/blob/08d3b6cdba887c92295b57b4475a0a3fae8fedb5/pkg/ccl/backupccl/restore_data_processor.go#L406) the opening of ssts associated with a restore span entry, it does not consider that backup file entries can be split mid key. As result, data corruption can occur.

As an example, suppose we want to restore aost `backupTime=8` and consider the two backup file entries `[a7,a5),[a4,a0)`: if the memory monitor preempts before opening `[a4,a0)`, then the restore data processor will add the backup key `a7` at `restoreTime=t0`, and then in the subsequent addsstable request at `restoreTime=t1`, will ingest backup key `a4`. So, after the restore finishes, backup key `a4` will be the latest key, but `a7` should be the latest key.

I think the easy fix here is to disallow memory monitoring preemption on an sst split mid key.

Jira issue: CRDB-30014

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.