cockroachdb / cockroachdb/cockroach

backup: teach `GetBackupManifests` to only read in required backup chain, instead of whole backup tree

Open
#140,645 1 comment 0 reactions 0 assignees View on GitHub
A-disaster-recovery C-enhancement P-3 T-disaster-recovery
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

With backup compactions enabled, a backup schedule could create 2x more inc backups per full backup. Given this, I’m worried that GetBackupManifests could OOM during Restore, incremental backup creation or compaction, as we’ll load all manifests in the backup tree, and then truncate them via ElideSkippedLayers. Ideally, we would only read the manifests necessary to build the linear chain we seek to use. Perhaps the only solution here is the backup directory rewrite project. As a stop gap, we can halve our recommended incs per full ratio.

Jira issue: CRDB-47280

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.