cockroachdb / cockroachdb/cockroach

backupccl: add option to restore to skip stats ingestion

Open
#101,054 2 comments 0 reactions 0 assignees View on GitHub
A-disaster-recovery C-enhancement T-disaster-recovery
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

One of the last stages of a restore is to insert the backed-up table statistics corresponding to the descriptors that have been restored. Currently, this phase issues inserts for each backed-up row serially. In situations where the backup has captured 1000s of table statistic rows (due to a large number of descriptors or a large number of columns per descriptor, or an accumulation of stale statistics) even if each insert takes on the order of 10s of ms, this can add a non-trivial amount of time to the restore. We have seen a recent support escalation where 29k table stats at ~70ms insert added ~40mins to the restore after it was done ingesting.

The restore also notifies the background stats refresher of all the newly created descriptors before the end of the restore, thereby triggering a new round of stats generation. Given all of this, a customer may be okay with degraded query performance on the restored tables, between when the restore completes and the background stats generation completes, at the benefit of the restore completing quicker. More often than not customers reaching for restore would choose temporary degraded performance over a restore job with a long tail. This long tail should be significantly improved by https://github.com/cockroachdb/cockroach/issues/101053 but it would be nice to have a `RESTORE WITH skip_table_statistics_insertion` option to reach for.

Jira issue: CRDB-26781

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.