Add CollectLeft partition mode to SortMergeJoinExec
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
SortMergeJoinExec currently always runs in a symmetric, hash-partitioned mode:
both inputs are hash-partitioned on the join keys and sorted, and partition i
of the left is merge-joined with partition i of the right.
When one side is small, hash-repartitioning the large side is wasteful. For
hash joins this is already handled by PartitionMode::CollectLeft (collect the
small build side once, broadcast it, don't repartition the probe side). This PR
brings the same idea to sort-merge joins.
### Describe the solution you'd like
Implement CollectLeft mode for SMJ
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start by tracing the existing PartitionMode::CollectLeft implementation for hash joins and the SortMergeJoinExec execution path. Verify how the small left side is collected and broadcast, then compare sort-merge join behavior with symmetric partitioning; done means CollectLeft avoids repartitioning the large side while producing correct join results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100