cockroachdb / cockroachdb/cockroach
backupccl: add option to restore to skip stats ingestion
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
One of the last stages of a restore is to insert the backed-up table statistics corresponding to the descriptors that have been restored. Currently, this phase issues inserts for each backed-up row serially. In situations where the backup has captured 1000s of table statistic rows (due to a large number of descriptors or a large number of columns per descriptor, or an accumulation of stale statistics) even if each insert takes on the order of 10s of ms, this can add a non-trivial amount of time to the restore. We have seen a recent support escalation where 29k table stats at ~70ms insert added ~40mins to the restore after it was done ingesting.
The restore also notifies the background stats refresher of all the newly created descriptors before the end of the restore, thereby triggering a new round of stats generation. Given all of this, a customer may be okay with degraded query performance on the restored tables, between when the restore completes and the background stats generation completes, at the benefit of the restore completing quicker. More often than not customers reaching for restore would choose temporary degraded performance over a restore job with a long tail. This long tail should be significantly improved by https://github.com/cockroachdb/cockroach/issues/101053 but it would be nice to have a `RESTORE WITH skip_table_statistics_insertion` option to reach for.
Jira issue: CRDB-26781
Contributor guide
Assessment
This issue has not been assessed yet.