cockroachdb / cockroachdb/cockroach

sql/inspect: regular-index consistency checks miss mixed-NULL entries

Open
#174,821 1 comment 0 reactions 1 assignee Claimed by @bowencrl View on GitHub
A-sql-execution C-bug O-agent T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

INSPECT can report success even when a regular secondary index has a missing or
dangling entry. This happens when its indexed and stored values mix NULL and
non-NULL values.

For example, consider an index on `b` that stores `e`. An entry with
`b = NULL, e = 'e_mixed'` falls into neither group used by the regular-index query:

- The non-NULL group requires `b IS NOT NULL AND e IS NOT NULL`.
- The NULL group requires `b IS NULL AND e IS NULL`.

Both conditions are false for that entry. The full check therefore skips it.
The partial-index query in PR [#3821](https://github.com/cockroachlabs/cockroach/pull/3821) uses an exhaustive any-NULL/all-non-NULL
split, but the regular-index query still uses the old split.

**To Reproduce**

At commit `5791dfe3eeaf5ca67580fe6e1cc2d96a70116a76`, add this test to
`pkg/sql/inspect/index_consistency_check_test.go`:

```go
func TestRegularIndexMixedNullReviewProbe(t *testing.T) {
defer leaktest.AfterTest(t)()
defer log.Scope(t).Close(t)
skip.UnderRace(t, "slow test")

runIndexConsistencyCheckTestCases(t, []indexConsistencyCheckTestCase{
{
desc: "missing mixed-NULL regular index entry",
indexDDL: []string{
"CREATE INDEX idx_t_regular ON test.t (b) STORING (e)",
},
postIndexSQL: "INSERT INTO test.t VALUES (5000, NULL, 600000, 'd_mixed', 'e_mixed', 1.5)",
missingIndexEntrySelector: "a = 5000",
expectedIssues: []inspectIssue{
{ErrorType: "missing_secondary_index_entry", PrimaryKey: "e'(5000, \\'d_mixed\\')'"},
},
expectedErrRegex: expectedInspectFoundInconsistencies,
},
{
desc: "dangling mixed-NULL regular index entry",
indexDDL: []string{
"CREATE INDEX idx_t_regular ON test.t (b) STORING (e)",
},
danglingIndexEntryInsertQuery: "SELECT 1502, NULL::INT, 300, 'ghostnull', 'e_x', 1.5",
expectedIssues: []inspectIssue{
{ErrorType: "dangling_secondary_index_entry", PrimaryKey: "e'(1502, \\'ghostnull\\')'"},
},
expectedErrRegex: expectedInspectFoundInconsistencies,
},
})
}
```

Run:

```sh
./dev test pkg/sql/inspect -f=TestRegularIndexMixedNullReviewProbe -v
```

This reuses the existing three-node corruption-test harness. The harness deletes
or inserts secondary-index KV entries directly, then runs
`INSPECT TABLE test.t ... WITH OPTIONS INDEX ALL`. Normal SQL writes maintain
the index and cannot create this corruption by themselves.

All four subtests fail with `An error is expected but got nil`:

| Corruption | Hash precheck enabled | Hash precheck disabled |
| --- | --- | --- |
| Missing entry, `b=NULL, e='e_mixed'` | INSPECT returns success | INSPECT returns success |
| Dangling entry, `b=NULL, e='e_x'` | INSPECT returns success | INSPECT returns success |

The hash precheck does not make this safe. A mismatch only triggers the full
query; it does not itself report the missing or dangling entry.

**Expected behavior**

Report `missing_secondary_index_entry` for the missing entry and
`dangling_secondary_index_entry` for the fabricated entry. Neither case should
return success.

**Additional data / screenshots**

The harness creates this table; both cases add the index shown below:

```sql
CREATE TABLE test.t (
a INT,
b INT,
c INT NOT NULL,
d TEXT,
e TEXT NOT NULL,
f FLOAT,
PRIMARY KEY (a, d),
FAMILY fam0 (a, b, c, d, e, f)
);
CREATE INDEX idx_t_regular ON test.t (b) STORING (e);
```

The regular-index split is in
[`createIndexCheckQuery`](https://github.com/cockroachlabs/cockroach/blob/5791dfe3eeaf5ca67580fe6e1cc2d96a70116a76/pkg/sql/inspect/index_consistency_check.go#L712).

**Environment:**

- CockroachDB source: `5791dfe3eeaf5ca67580fe6e1cc2d96a70116a76`, PR [#3821](https://github.com/cockroachlabs/cockroach/pull/3821).
- Local test date: 2026-09-07.
- Server OS: macOS, arm64; in-process Go test cluster.
- Client: the package's existing SQL/KV corruption-test harness.
- Earliest affected release has not been verified.

**Additional context**

This is a corruption-detection gap, not a mechanism that creates corruption.
It is separate from the partial-index physical-scan limitations.

Epic CRDB-65904

Jira issue: CRDB-68015

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.