try to add local limit after the optimizer rule `join_selection`
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
In our cases, we have left_table(multi file group) and right_table(single file group), and we write the sql
```
select * from left_table join right_table on left_table.key = right_table.key
```
The final physical plan is not the optimal.
The init phy plan is below with **auto** join mode. The children of the join will be swapped by the rule `join_selection`
```
GlobalLimitExec: skip=0, fetch=10
ProjectionExec: ....
HashJoinExec: mode=**auto**, join_type=Inner, on=....
ProjectionExec: left_table
ParquetExec: file_groups={37 group: ....}
ProjectionExec: right_table
ParquetExec: file_groups={1 groups: ....}
```
after the physical optimizer, the most import rule is `join_selection`
The final plan is like:
```
....
GlobalLimitExec: skip=0, fetch=10
ProjectionExec: ....
HashJoinExec: **mode=CollectLeft,** join_type=Inner, on=....
ProjectionExec: right_table
ParquetExec: file_groups={1 group: ....}
ProjectionExec: left_table
ParquetExec: file_groups={37 groups: ....}
```
The optimal plan should be added a `LocalLimitExec` behind `GlobalLimitExec`
### Describe the solution you'd like
The optimal plan should be like
```
....
GlobalLimitExec: skip=0, fetch=10
LocalLimitExec:....
ProjectionExec: ....
HashJoinExec: **mode=CollectLeft,** join_type=Inner, on=....
ProjectionExec: right_table
ParquetExec: file_groups={1 group: ....}
ProjectionExec: left_table
ParquetExec: file_groups={37 groups: ....}
```
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start by tracing the physical optimizer rule `join_selection` and the `GlobalLimitExec` and `LocalLimitExec` plan nodes shown in the issue. Reproduce the SQL plan with a multi-file left table and single-file right table, then verify that the optimized plan places a local limit below the global limit while retaining the selected join mode.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100