Allow for bounds analysis on selective columns in a schema
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
When performing bounds analysis on a filter expression (`Expr`), I would like to only analyze bounds for specific columns in my schema, and ignore other columns that are in the `Expr`. However, the [`analyze`](https://github.com/apache/datafusion/blob/7c07948358eac81c4b297fa2400cba3c9ca55dc2/datafusion/physical-expr/src/analysis.rs#L159) and underlying [`ExprIntervalGraph`](https://github.com/apache/datafusion/blob/7c07948358eac81c4b297fa2400cba3c9ca55dc2/datafusion/physical-expr/src/intervals/cp_solver.rs#L123) analyzes bounds for all nodes in the `Expr` tree.
The problem is that sometimes the `Expr` tree can have a mismatch in the LHS/RHS data type, which leads to the analysis failing due to errors like [this one](https://github.com/apache/datafusion/blob/7c07948358eac81c4b297fa2400cba3c9ca55dc2/datafusion/expr-common/src/interval_arithmetic.rs#L1195-L1199).
### Describe the solution you'd like
When analyzing an expression like
```
WHERE a < 10 AND b > "aaa"
```
I would like there to be a way to only analyze the bounds imposed on `a`, which in the above should result in the interval [`None`, `9`].
### Describe alternatives you've considered
I may not be using the correct APIs, or am using the correct APIs incorrectly; if either are the case, then perhaps I can learn what is correct and create an example for my use case.
There may be other solutions to this problem that I am not aware of. DataFusion may also be doing expression simplification with respect to the schema that make this less of a problem than it appears to be in my contrived [reproducer](https://github.com/apache/datafusion/pull/14308).
### Additional context
I opened a reproducer for this here: https://github.com/apache/datafusion/pull/14308
Contributor guide
Research direction
Start with analyze in datafusion/physical-expr/src/analysis.rs and ExprIntervalGraph in datafusion/physical-expr/src/intervals/cp_solver.rs. Review reproducer PR 14308 and the interval arithmetic error location in datafusion/expr-common/src/interval_arithmetic.rs; done should include selective analysis of a while expression that returns [None, 9] for a and ignores the incompatible b condition.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100