Redundant `IS NOT NULL` predicate is not eliminated for `BETWEEN`
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
In a `WHERE` predicate, `a BETWEEN 1 AND 3` already filters out `NULL` values because the expression evaluates to `UNKNOWN` for `NULL`, and `UNKNOWN` does not pass the `WHERE` filter.
Therefore, the following two predicates are semantically equivalent:
```sql
a BETWEEN 1 AND 3
```
and:
```sql
a BETWEEN 1 AND 3 AND a IS NOT NULL
```
However, TiDB keeps the redundant `not(isnull(a))` condition in the physical plan instead of eliminating it.
This is visible directly from `EXPLAIN`, so the issue can be reproduced stably without relying on runtime noise.
### 1. Minimal reproduce step (Required)
```sql
DROP DATABASE IF EXISTS rift_between_not_null;
CREATE DATABASE rift_between_not_null;
USE rift_between_not_null;
CREATE TABLE t (
id INT PRIMARY KEY,
a INT,
b VARCHAR(20)
);
INSERT INTO t VALUES
(1, 1, 'a'),
(2, 2, 'b'),
(3, 4, 'c'),
(4, NULL, 'd');
EXPLAIN SELECT * FROM t WHERE a BETWEEN 1 AND 3;
EXPLAIN SELECT * FROM t WHERE a BETWEEN 1 AND 3 AND a IS NOT NULL;
```
### 2. What did you expect to see? (Required)
TiDB should remove the redundant `IS NOT NULL` predicate when it is already implied by a `BETWEEN` predicate in a `WHERE` filter.
The optimized plan should be equivalent to:
```text
Selection ... ge(t.a, 1), le(t.a, 3)
```
### 3. What did you see instead (Required)
For:
```sql
EXPLAIN SELECT * FROM t WHERE a BETWEEN 1 AND 3;
```
TiDB produces:
```text
Selection ... ge(rift_between_not_null.t.a, 1), le(rift_between_not_null.t.a, 3)
```
For:
```sql
EXPLAIN SELECT * FROM t WHERE a BETWEEN 1 AND 3 AND a IS NOT NULL;
```
TiDB produces:
```text
Selection ... ge(rift_between_not_null.t.a, 1), le(rift_between_not_null.t.a, 3), not(isnull(rift_between_not_null.t.a))
```
The extra `not(isnull(...))` predicate is redundant.
### 4. What is your TiDB version? (Required)
```text
Release Version: v8.5.7
Edition: Community
Git Commit Hash: 202b7f47286a1109b5c957401d34c9358d130ae0
Git Branch: HEAD
UTC Build Time: 2026-07-15 02:06:00
GoVersion: go1.25.10
Race Enabled: false
Check Table Before Drop: false
Store: unistore
```
Contributor guide
Research direction
Start by running the SQL reproduction and comparing the two EXPLAIN outputs, then trace TiDB's optimizer handling for BETWEEN and IS NOT NULL. Add a regression test for the redundant predicate and verify that the physical plan no longer includes not(isnull(...)) while preserving the expected range conditions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 66/100