[FEA] Simplify null pre-processing in sort-merge join
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
The lexicographic row comparator assumes null entries to be equivalent regardless of how nulls are to be compared in the join operation, which results in incorrect join output when `null_equality::UNEQUAL` is set. To circumvent this issue, the current sort-merge join implementation filters the input tables to remove all rows with null entries at the root level - and children level in case of lists - which can have a large memory footprint if the number of null rows is small.
**Describe the solution you'd like**
Support null comparison in lexicographic row operators based on the null equality set. A significant chunk of the pre-processing logic for sort-merge join can then be removed.
Contributor guide
Assessment
This issue has not been assessed yet.