apache / apache/datafusion

Do dynamic filters (e.g. from SortExec) work with recursive queries?

Open
#17,060 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

We're not sure they ever worked correctly, but it's possible https://github.com/apache/datafusion/pull/17028 broke them entirely. We should confirm. I have not however been able to come up with a reasonable example, recursive queries are not something I use often enough to have a good intuition for. See https://github.com/apache/datafusion/pull/17028 for discussion.

Contributor guide

Open the contributing guide

Research direction

Start by reading the discussion in pull request 17028 and trace how dynamic filters from SortExec are handled during recursive queries. Create a reasonable recursive-query example to determine whether the filters work correctly; done means the behavior is confirmed and covered by an appropriate regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.