cockroachdb / cockroachdb/cockroach

backfill: use INSPECT to validate new indexes created in schema change backfill

Open
#156,582 2 comments 0 reactions 0 assignees View on GitHub
C-enhancement O-support P-3 T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Is your feature request related to a problem? Please describe.**
After an index backfill, we run validation on the newly created index to make sure it has the expected number of rows. See https://github.com/cockroachdb/cockroach/blob/07d4a960e29fdaf29fc7715835e3223e272615b1/pkg/sql/backfill.go#L1874.

This validation executes one large SELECT query to count all the rows in the index. This query can take very long for large tables, and there is no easy way to see the progress of that query or come up with an estimate of how long it will take. Also, if the job gets restarted during this phase, it means the SELECT needs to be re-executed and start over from the beginning.

**Describe the solution you'd like**
The new INSPECT command basically does the same thing -- it compares index counts. It also will soon have the ability to validate row count based on an expected number: https://github.com/cockroachdb/cockroach/issues/155472

The backfill validation code could use INSPECT, and direct users to view the progress of the INSPECT job to see how much longer validation will take.

**Describe alternatives you've considered**
Instead of one large SELECT, we could break up the work into smaller chunks. But this would basically duplicate a lot of the logic that INSPECT has already.

**Additional context**
Came up in a support case https://github.com/cockroachlabs/support/issues/3469

Jira issue: CRDB-56069

Epic CRDB-55075

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.