[C++][Dataset] Provide more robust handling of comparison guarantees in the presence of implicit casts
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
After ARROW-8919 it's possible that a field reference may be wrapped in an implicit cast, which complicates destructuring during expression simplification. In particular, some errors can arise as a result of assuming that casts will preserve numeric ordering:
```c++
int two_28 = 1 << 28;
auto partition_expr = less_equal(field_ref("i32"), literal(two_28 + 1));
auto filter = greater(
cast(field_ref("i32"), float32()),
literal(float(two_28)));
```
Currently the RHS of the filter and the partition expression will be considered equal since casting `two_28+1` to float results in the same value as casting `two_28` to float (due to limited FP precision). Since `x <= y` and `x > y` are disjoint, the partition will skipped entirely including any rows where `i32 == two_28+1` (which **should** be selected by this filter).
**Reporter**: [Ben Kietzman](https://issues.apache.org/jira/browse/ARROW-11562) / @bkietz
**Note**: *This issue was originally created as [ARROW-11562](https://issues.apache.org/jira/browse/ARROW-11562). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by reproducing the C++ example in the issue and tracing expression simplification and destructuring around the implicit-cast behavior introduced by ARROW-8919. The fix is done when the partition expression is not treated as equal to the filter merely because floating-point casts collapse distinct integer values, and rows with i32 == two_28+1 remain selectable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100