NVIDIA / NVIDIA/cudf

[FEA] Support three-valued (Kleene) logic in any and all aggregations

Open
#16,475 0 comments 0 reactions 0 assignees View on GitHub
feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

In the presence of null values in a boolean column, many libraries support both null-propagating two-valued logic and null-including three-valued (Kleene) logic. For example, this is the case in both polars and spark.

For pointwise binary operations, libcudf offers the `NULL_LOGICAL_AND` and `NULL_LOGICAL_OR` operators which implement the relevant three-way truth tables.

However, if we wish to perform an aggregation with this same operator, it is not possible to achieve the same result without introspective post-processing of the return values.

**Describe the solution you'd like**

I'd like to be able to provide a `null_policy` argument to creation of `any` and `all` aggregations. If `null_policy == INCLUDE`, the relevant three-valued truth table should be used for the aggregation, if `null_policy == EXCLUDE` (which can be the default), the current behaviour should be maintained.

**Describe alternatives you've considered**

Post-processing the result of an aggregation is possible, but gets messy. For a whole column this is OK, though necessitates a device->host transfer to obtain the result, and then a call to `scalar.set_valid_async` producing a host->device transfer.

In a grouped or rolling context it is much harder since that same checking must be made for each result group, determining which group contained nulls before the aggregation, and combining the groupwise results with the validity in the appropriate cases.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.