cockroachdb / cockroachdb/cockroach

opt: partial index implicator cannot prove NOT IN predicate implication

Open
#166,728 4 comments 0 reactions 0 assignees View on GitHub
A-partial-indexes A-sql-optimizer C-performance O-community T-sql-queries X-blathers-triaged
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

## Description

The partial index implicator cannot prove that a `NOT IN` filter in a query implies a `NOT IN` predicate on a partial index, **but only for INT2 and INT4 columns**. STRING, INT8, and DECIMAL columns work correctly.

## Repro

```sql
CREATE TABLE orders (
id INT8 PRIMARY KEY,
parent_id INT8,
status INT2 NOT NULL
);

INSERT INTO orders VALUES (1, 100, 1), (2, 100, 5), (3, 100, 7);

-- Create partial index with NOT IN predicate on an INT2 column
CREATE INDEX orders_pending_idx ON orders (parent_id)
WHERE parent_id IS NOT NULL AND status NOT IN (5, 7);

-- Check stored predicate — note the INT8 literals on an INT2 column:
SHOW CREATE TABLE orders;
-- INDEX orders_pending_idx (parent_id ASC)
-- WHERE (parent_id IS NOT NULL) AND (status NOT IN (5:::INT8, 7:::INT8))

-- This fails:
SELECT * FROM orders@orders_pending_idx
WHERE parent_id IS NOT NULL AND status NOT IN (5, 7);
-- ERROR: partial index does not contain all the rows needed

-- Workaround: explicit INT2 casts in the query make it work:
SELECT * FROM orders@orders_pending_idx
WHERE parent_id IS NOT NULL AND status NOT IN (5::INT2, 7::INT2);
-- OK!
```

The same pattern works fine for types where the literal type matches the column type:

```sql
-- INT8 column: works (literals are already INT8)
CREATE TABLE t_int8 (id INT8 PRIMARY KEY, status INT8 NOT NULL);
CREATE INDEX t_int8_idx ON t_int8 (id) WHERE status NOT IN (5, 7);
SELECT * FROM t_int8@t_int8_idx WHERE status NOT IN (5, 7); -- OK

-- STRING column: works (string literals match VARCHAR)
CREATE TABLE t_str (id INT8 PRIMARY KEY, status VARCHAR NOT NULL);
CREATE INDEX t_str_idx ON t_str (id) WHERE status NOT IN ('X', 'Y');
SELECT * FROM t_str@t_str_idx WHERE status NOT IN ('X', 'Y'); -- OK

-- DECIMAL column: works
CREATE TABLE t_dec (id INT8 PRIMARY KEY, x DECIMAL NOT NULL);
CREATE INDEX t_dec_idx ON t_dec (x) WHERE x NOT IN (5, 7);
SELECT * FROM t_dec@t_dec_idx WHERE x NOT IN (5, 7); -- OK

-- INT4 column: fails (same as INT2)
CREATE TABLE t_int4 (id INT8 PRIMARY KEY, status INT4 NOT NULL);
CREATE INDEX t_int4_idx ON t_int4 (id) WHERE status NOT IN (5, 7);
SELECT * FROM t_int4@t_int4_idx WHERE status NOT IN (5, 7); -- ERROR
```

Tested on v23.2.2, v24.3.17, and v25.4.5 — same behavior on all three.

## Observations from the Code

The implicator (`pkg/sql/opt/partialidx/implicator.go`) uses two strategies to prove that a query filter implies an index predicate: expression comparison and constraint-set containment.

For `NOT IN` with multiple elements, the constraint builder (`pkg/sql/opt/memo/constraint_builder.go`) has no handler — `InOp` has a special case at line 72 that builds span constraints, but there is no parallel case for `NotInOp`. Multi-element `NOT IN` falls through to `unconstrained`, so the constraint-based comparison path always fails.

That leaves expression comparison as the only path. For INT8 and STRING columns, the literal types in the index predicate naturally match the literal types in the query filter, so expression comparison succeeds. For INT2/INT4, a normalization divergence causes the expression trees to differ — the stored predicate has INT8 literals (`5:::INT8, 7:::INT8` as shown by `SHOW CREATE TABLE`), and the query filter is normalized differently. The `::INT2` cast workaround forces the query-side expression to match.

Single-element `NOT IN` avoids this because `SimplifyNotInSingleElement` (`pkg/sql/opt/norm/rules/scalar.opt`) converts it to `!=`, and then `UnifyComparisonTypes` casts the constant to the column type.

## Practical Impact

INT2 is commonly used for status/type columns in application schemas. Partial indexes that exclude terminal states (`WHERE status NOT IN (done, cancelled)`) are a natural pattern for these columns. The current behavior forces users to either:
- Use `IN` with the full complement set (verbose, breaks silently when new values are added)
- Use explicit `::INT2` casts in every query (fragile, requires application awareness of the DB column type)

Both workarounds are error-prone.

Related: #122461

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.