cockroachdb / cockroachdb/cockroach
sql/inspect: regular-index consistency checks miss mixed-NULL entries
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
INSPECT can report success even when a regular secondary index has a missing or
dangling entry. This happens when its indexed and stored values mix NULL and
non-NULL values.
For example, consider an index on `b` that stores `e`. An entry with
`b = NULL, e = 'e_mixed'` falls into neither group used by the regular-index query:
- The non-NULL group requires `b IS NOT NULL AND e IS NOT NULL`.
- The NULL group requires `b IS NULL AND e IS NULL`.
Both conditions are false for that entry. The full check therefore skips it.
The partial-index query in PR [#3821](https://github.com/cockroachlabs/cockroach/pull/3821) uses an exhaustive any-NULL/all-non-NULL
split, but the regular-index query still uses the old split.
**To Reproduce**
At commit `5791dfe3eeaf5ca67580fe6e1cc2d96a70116a76`, add this test to
`pkg/sql/inspect/index_consistency_check_test.go`:
```go
func TestRegularIndexMixedNullReviewProbe(t *testing.T) {
defer leaktest.AfterTest(t)()
defer log.Scope(t).Close(t)
skip.UnderRace(t, "slow test")
runIndexConsistencyCheckTestCases(t, []indexConsistencyCheckTestCase{
{
desc: "missing mixed-NULL regular index entry",
indexDDL: []string{
"CREATE INDEX idx_t_regular ON test.t (b) STORING (e)",
},
postIndexSQL: "INSERT INTO test.t VALUES (5000, NULL, 600000, 'd_mixed', 'e_mixed', 1.5)",
missingIndexEntrySelector: "a = 5000",
expectedIssues: []inspectIssue{
{ErrorType: "missing_secondary_index_entry", PrimaryKey: "e'(5000, \\'d_mixed\\')'"},
},
expectedErrRegex: expectedInspectFoundInconsistencies,
},
{
desc: "dangling mixed-NULL regular index entry",
indexDDL: []string{
"CREATE INDEX idx_t_regular ON test.t (b) STORING (e)",
},
danglingIndexEntryInsertQuery: "SELECT 1502, NULL::INT, 300, 'ghostnull', 'e_x', 1.5",
expectedIssues: []inspectIssue{
{ErrorType: "dangling_secondary_index_entry", PrimaryKey: "e'(1502, \\'ghostnull\\')'"},
},
expectedErrRegex: expectedInspectFoundInconsistencies,
},
})
}
```
Run:
```sh
./dev test pkg/sql/inspect -f=TestRegularIndexMixedNullReviewProbe -v
```
This reuses the existing three-node corruption-test harness. The harness deletes
or inserts secondary-index KV entries directly, then runs
`INSPECT TABLE test.t ... WITH OPTIONS INDEX ALL`. Normal SQL writes maintain
the index and cannot create this corruption by themselves.
All four subtests fail with `An error is expected but got nil`:
| Corruption | Hash precheck enabled | Hash precheck disabled |
| --- | --- | --- |
| Missing entry, `b=NULL, e='e_mixed'` | INSPECT returns success | INSPECT returns success |
| Dangling entry, `b=NULL, e='e_x'` | INSPECT returns success | INSPECT returns success |
The hash precheck does not make this safe. A mismatch only triggers the full
query; it does not itself report the missing or dangling entry.
**Expected behavior**
Report `missing_secondary_index_entry` for the missing entry and
`dangling_secondary_index_entry` for the fabricated entry. Neither case should
return success.
**Additional data / screenshots**
The harness creates this table; both cases add the index shown below:
```sql
CREATE TABLE test.t (
a INT,
b INT,
c INT NOT NULL,
d TEXT,
e TEXT NOT NULL,
f FLOAT,
PRIMARY KEY (a, d),
FAMILY fam0 (a, b, c, d, e, f)
);
CREATE INDEX idx_t_regular ON test.t (b) STORING (e);
```
The regular-index split is in
[`createIndexCheckQuery`](https://github.com/cockroachlabs/cockroach/blob/5791dfe3eeaf5ca67580fe6e1cc2d96a70116a76/pkg/sql/inspect/index_consistency_check.go#L712).
**Environment:**
- CockroachDB source: `5791dfe3eeaf5ca67580fe6e1cc2d96a70116a76`, PR [#3821](https://github.com/cockroachlabs/cockroach/pull/3821).
- Local test date: 2026-09-07.
- Server OS: macOS, arm64; in-process Go test cluster.
- Client: the package's existing SQL/KV corruption-test harness.
- Earliest affected release has not been verified.
**Additional context**
This is a corruption-detection gap, not a mechanism that creates corruption.
It is separate from the partial-index physical-scan limitations.
Epic CRDB-65904
Jira issue: CRDB-68015
Contributor guide
Assessment
This issue has not been assessed yet.