Optimize `try_embed_to_hash_join` in the ProjectionPushdown rule
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
From https://github.com/apache/arrow-datafusion/pull/9236/files#r1513524992,
This is a TODO in datafusion/core/src/physical_optimizer/optimizer.rs:
> TODO: `try_embed_to_hash_join` in the ProjectionPushdown rule would be block by the CoalesceBatches, so add it before CoalesceBatches. Maybe optimize it in the future.
We can optimize `try_embed_to_hash_join` in `ProjectionPushdown` rule to make it unblock by `CoalesceBatches`.
### Describe the solution you'd like
_No response_
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Read datafusion/core/src/physical_optimizer/optimizer.rs and locate try_embed_to_hash_join in the ProjectionPushdown rule. Trace how CoalesceBatches currently blocks it and review the surrounding optimizer flow. Done means the function is optimized so it is no longer blocked by CoalesceBatches, with the relevant optimizer behavior verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100