apache / apache/datafusion

Additional `IN` / `NOT IN` simplifications

Open
#8,970 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

In the following on PR, I think we can move inlist related simplifier logic to `inlist_simplifier`.
Steps in inlist_simplifier
1. early stage simplification like `expr IN () --> false`, `null IN/NOT IN`
2. convert long OR chain or AND chain to InList expressions like what `or_in_list_simplifier` does
3. shorten InList expressions like what `inlist_simplifier` in this PR does
4. convert final inlist to OR chain or AND chain if len < 3. Move inlist related simplifier out from `expr_simplifier`

In this way, when the logic grows more complex, we can easily find out related conversion about specific Expr

_Originally posted by @jayzhan211 in https://github.com/apache/arrow-datafusion/issues/8949#issuecomment-1906115890_

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing expr_simplifier, inlist_simplifier, and or_in_list_simplifier logic to trace how IN and NOT IN expressions are handled. The refactor is done when the listed early simplification, chain conversion, shortening, and final expansion steps are organized through inlist_simplifier without inlist-specific logic remaining in expr_simplifier.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.