cockroachdb / cockroachdb/cockroach
backfill: use INSPECT to validate new indexes created in schema change backfill
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Is your feature request related to a problem? Please describe.**
After an index backfill, we run validation on the newly created index to make sure it has the expected number of rows. See https://github.com/cockroachdb/cockroach/blob/07d4a960e29fdaf29fc7715835e3223e272615b1/pkg/sql/backfill.go#L1874.
This validation executes one large SELECT query to count all the rows in the index. This query can take very long for large tables, and there is no easy way to see the progress of that query or come up with an estimate of how long it will take. Also, if the job gets restarted during this phase, it means the SELECT needs to be re-executed and start over from the beginning.
**Describe the solution you'd like**
The new INSPECT command basically does the same thing -- it compares index counts. It also will soon have the ability to validate row count based on an expected number: https://github.com/cockroachdb/cockroach/issues/155472
The backfill validation code could use INSPECT, and direct users to view the progress of the INSPECT job to see how much longer validation will take.
**Describe alternatives you've considered**
Instead of one large SELECT, we could break up the work into smaller chunks. But this would basically duplicate a lot of the logic that INSPECT has already.
**Additional context**
Came up in a support case https://github.com/cockroachlabs/support/issues/3469
Jira issue: CRDB-56069
Epic CRDB-55075
Contributor guide
Assessment
This issue has not been assessed yet.