[FEA] Refactor various conditional join implementations for simplicity and API consistency
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
#9917 and #10037 added new join functions that use a mixture of hash lookups and AST expression evaluation. In the interest of time, a number of important performance improvements, API design questions, and general internal refactorings were overlooked. This issue aims to track those potential improvements for future work.
**Describe the solution you'd like**
- [x] Implement benchmarks for mixed joins. This is a critical first step to evaluate the importance of other changes.
- [ ] Implement object-oriented APIs for mixed joins.
- [ ] Remove size APIs for semi/anti joins (both mixed and pure conditional) in favor of a single-kernel approach. The size APIs aren't really necessary since the primary use case for those is knowing when the results will be large enough to spill in multi-GPU cases, and for semi/anti joins the size is bounded by just the number of rows a table N (rather than N^2). It would be simpler and more efficient to use an approach like the hash semi/anti joins, which essentially just generate a gather mask.
- [x] #15250 for mixed semi/anti-joins
- [x] https://github.com/rapidsai/cudf/pull/14646#discussion_r1568113565 for conditional semi/anti-joins
- [x] Rework conditional join internals for semi/anti joins to use kernels that don't allocate the second output vector, which is wasteful.
- [x] https://github.com/rapidsai/cudf/pull/14646
- [ ] Find cleaner solutions for the expression evaluator shared memory handling, see https://github.com/rapidsai/cudf/pull/9917#discussion_r779021242.
- [ ] Reduce compile time if possible (https://github.com/rapidsai/cudf/pull/9917#discussion_r771559204).
- [ ] Explore more code sharing (perhaps via templating) between mixed and pure conditional joins, as well as more sharing with the existing hash join infrastructure (should be facilitated by implementing the object-oriented API).
- [ ] Revisit naming of different join APIs. In the long term the joins probably shouldn't include names based on whether they are equality, conditional, or mixed joins, they should just be named by the type of join (inner, left, etc) and rely on the signature to differentiate the rest.
Contributor guide
Assessment
This issue has not been assessed yet.