crate / crate/crate

Add rule to filter for null values at inner-joins

Open
#14,721 9 comments 2 reactions 0 assignees View on GitHub
complexity: no estimate feature: performance
Dominant language
Java
Stars
4.4k
Forks
616
Avg merge
14h 49m
Merged PRs (30d)
206

Description

Assume the following query:

```
SELECT * FROM t1 INNER JOIN t2 ON t1.a = t2.b
```

The `t2.b` column does not allow null values but `t1.a` does. Rows in `t1` where `a` is null can never match any row in `t2.b` for the inner join case with an equi-join condition `t1.a = t2.b`. Therefore, we can add a filter when reading `t1` to filter out rows with null values before we perform the join to reduce the number of rows.

```
Inner-Join(t1.a = t2.b)
/ \
t1.a t2.b
```
becomes:
```
Inner-Join(t1.a = t2.b)
/ \
Filter(a is not null) \
/ \
t1.a t2.b
```

### Reference:
https://github.com/apache/arrow-datafusion/blob/main/datafusion/optimizer/src/filter_null_join_keys.rs

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.