PushDownFilter optimizer pushes down filters through distinct on
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
a filter should not be pushed down through a distinct on because it will change the results.
The PushDownFilter optimizer rule does not behave correctly.
### To Reproduce
Add this test to push_down_filter.rs:
```
#[test]
fn distinct_on() -> Result<()> {
let table_scan = test_table_scan()?;
let plan = LogicalPlanBuilder::from(table_scan)
.distinct_on(vec![col("a")], vec![col("a")], None)?
.filter(col("a").eq(lit(1i64)))?
.build()?;
// filter appears below Union
let expected = "\
Filter: a = Int64(1)\
\n DistinctOn: on_expr=[[test.a]], select_expr=[[a]], sort_expr=[[]]\
\n TableScan: test";
assert_optimized_plan_eq(plan, expected)
}
```
you will get the logical plan:
```
DistinctOn: on_expr=[[test.a]], select_expr=[[a]], sort_expr=[[]]
TableScan: test, full_filters=[a = Int64(1)]
```
### Expected behavior
Filters should not be pushed down through distinct on.
the logical plan should be as expected in the above test.
### Additional context
_No response_
Contributor guide
Research direction
Start in push_down_filter.rs and add the distinct_on test shown in the issue. Run the relevant optimizer tests to reproduce the current plan and compare it with the expected plan. Done means the filter remains above DistinctOn and the test passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 50/100