Add rule to filter for null values at inner-joins
- Dominant language
- Java
- Stars
- 4.4k
- Forks
- 616
- Avg merge
- 14h 49m
- Merged PRs (30d)
- 206
Description
Assume the following query:
```
SELECT * FROM t1 INNER JOIN t2 ON t1.a = t2.b
```
The `t2.b` column does not allow null values but `t1.a` does. Rows in `t1` where `a` is null can never match any row in `t2.b` for the inner join case with an equi-join condition `t1.a = t2.b`. Therefore, we can add a filter when reading `t1` to filter out rows with null values before we perform the join to reduce the number of rows.
```
Inner-Join(t1.a = t2.b)
/ \
t1.a t2.b
```
becomes:
```
Inner-Join(t1.a = t2.b)
/ \
Filter(a is not null) \
/ \
t1.a t2.b
```
### Reference:
https://github.com/apache/arrow-datafusion/blob/main/datafusion/optimizer/src/filter_null_join_keys.rs
Contributor guide
Assessment
This issue has not been assessed yet.