apache / apache/datafusion

regression in optimizing filter push down of CAST + BETWEEN

Open
#3,863 1 comment 0 reactions 0 assignees View on GitHub
bug optimizer
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

**Describe the bug**

There has been a regression (since a92119a0584322234a9005dc0897e94026464cad) in pushing BETWEEN expressions down to table scans.

## before

Table scan filter: CAST(col1 AS Int64) BETWEEN Int64(4) AND Int64(4) + Int64(3)

## after

Table scan filter: CAST(col1 AS Int64) >= Int64(4), CAST(col1 AS Int64) <= Int64(7)

Performing the CAST twice is not efficient.

**To Reproduce**
Steps to reproduce the behavior:

**Expected behavior**
A clear and concise description of what you expected to happen.

**Additional context**
Add any other context about the problem here.

Contributor guide

Open the contributing guide

Research direction

Review regression commit a92119a0584322234a9005dc0897e94026464cad and trace the filter-pushdown path for CAST combined with BETWEEN. Reproduce the reported table-scan filter change and confirm that the optimized result avoids evaluating the CAST twice while preserving the BETWEEN bounds.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.