cockroachdb / cockroachdb/cockroach
Online Restore Recovery: Torn Transaction Records
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
As of [PR #144244](https://github.com/cockroachdb/cockroach/pull/144244), online restore supports deleting restoring tables that reference corrupt or unavailable remote SSTs.
In version 25.2, only "offline" online restore will be supported—descriptors remain offline until the download completes. Full online restore will be supported in the future, but it introduces complications with the recovery protocol.
Currently, [a transaction record is stored in the first range written by the transaction](https://www.cockroachlabs.com/docs/v24.3/architecture/transaction-layer#writing). If that range becomes unavailable and is excised, its transaction record is deleted. This is typically safe because the associated data is also removed. However, problems arise if the first write is to a table restored online, and later writes target tables outside the restore.
Example:
t0: transaction record, row, and intent written to range A
t1: row and intent written to range B
t2: row and intent written to range C
t3: transaction is committed
t4: intent on range B is resolved
t5: range A is excised
In this case, the row in range B is committed, but the data in range C is rolled back.
A similar issue occurs with survive zone and survive region tables. A transaction recorded on a survive zone range can cover writes to a survive region range. Both issues could be addressed simultaneously: if a transaction spans ranges with varying durability (e.g., remote survive zone < survive zone < remote survive region < survive region), the transaction record should be stored in the most durable range.
Jira issue: CRDB-49370
Epic CRDB-50824
Contributor guide
Assessment
This issue has not been assessed yet.