[FEA] Support three-valued (Kleene) logic in any and all aggregations
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
In the presence of null values in a boolean column, many libraries support both null-propagating two-valued logic and null-including three-valued (Kleene) logic. For example, this is the case in both polars and spark.
For pointwise binary operations, libcudf offers the `NULL_LOGICAL_AND` and `NULL_LOGICAL_OR` operators which implement the relevant three-way truth tables.
However, if we wish to perform an aggregation with this same operator, it is not possible to achieve the same result without introspective post-processing of the return values.
**Describe the solution you'd like**
I'd like to be able to provide a `null_policy` argument to creation of `any` and `all` aggregations. If `null_policy == INCLUDE`, the relevant three-valued truth table should be used for the aggregation, if `null_policy == EXCLUDE` (which can be the default), the current behaviour should be maintained.
**Describe alternatives you've considered**
Post-processing the result of an aggregation is possible, but gets messy. For a whole column this is OK, though necessitates a device->host transfer to obtain the result, and then a call to `scalar.set_valid_async` producing a host->device transfer.
In a grouped or rolling context it is much harder since that same checking must be made for each result group, determining which group contained nulls before the aggregation, and combining the groupwise results with the validity in the appropriate cases.
Contributor guide
Assessment
This issue has not been assessed yet.