apache / apache/datafusion

Nullable expression over non-null columns are planned as non-null-aware-joins

Open
#25,024 1 comment 0 reactions 1 assignee Claimed by @AdamGS View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

As seen in the query below, we currently only take column nullability into consideration when planning joins instead of full expression nullability, in cases where we join on an expression that might introduce nulls.

### To Reproduce

```sql
CREATE TABLE outer_t(id VARCHAR NOT NULL) AS VALUES ('z');
CREATE TABLE inner_t(id VARCHAR NOT NULL) AS VALUES ('a');

SELECT id
FROM outer_t
WHERE (NULLIF(id, 'z') NOT IN (SELECT id FROM inner_t)) IS NULL;
```

### Expected behavior

The query above currently returns no results, but should return a single row with the value `z` (verified with DuckDB).

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.