NVIDIA / NVIDIA/cudf

[FEA] Simplify null pre-processing in sort-merge join

Open
#19,268 0 comments 0 reactions 0 assignees View on GitHub
feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
The lexicographic row comparator assumes null entries to be equivalent regardless of how nulls are to be compared in the join operation, which results in incorrect join output when `null_equality::UNEQUAL` is set. To circumvent this issue, the current sort-merge join implementation filters the input tables to remove all rows with null entries at the root level - and children level in case of lists - which can have a large memory footprint if the number of null rows is small.

**Describe the solution you'd like**
Support null comparison in lexicographic row operators based on the null equality set. A significant chunk of the pre-processing logic for sort-merge join can then be removed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.