Consider deprecation (or removal) of `PhysicalExpr::evaluate_selection`
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Recent changes in the implementation of `CaseExpr` have replaced the usages of `PhysicalExpr::evaluate_selection` with custom filtering and selective evaluation logic. This was done to avoid overhead introduced by the generic approach `evaluate_selection` takes where it first filters the incoming record batch using the selection vector, calls plain `evaluate` with the filtered record batch, and then expands the result back to the length of the original record batch.
Because of all these changes, `PhysicalExpr::evaluate_selection` is no longer used at all in DataFusion itself. Since it's no longer used, and its use can be a cause of performance overhead, it might be better to deprecate (or even remove) it.
### Describe the solution you'd like
_No response_
### Describe alternatives you've considered
_No response_
### Additional context
`evaluate_selection` was first introduced in #2068
As can be seen from the discussion at https://github.com/apache/arrow-rs/issues/3620 actually implementing selective evaluation more efficiently is not trivial. Does it make sense to have this as an overridable trait function?
Contributor guide
Research direction
Start by locating the PhysicalExpr::evaluate_selection trait method and checking whether any DataFusion implementations or call sites remain; then inspect the CaseExpr selective-evaluation changes described in the issue. Done means reaching and documenting a clear decision on deprecation versus removal, including any affected API users or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100