[FEA] Support predicated aggregations in libcudf
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
## Ask
Add libcudf support for predicated aggregations, where an aggregation is evaluated only over rows selected by a boolean predicate without first materializing a filtered intermediate column.
## Motivation
Higher-level APIs (e.g. Polars) can express groupby aggregations over filtered values, for example:
```python
df.group_by("key").agg(
pl.col("value").filter(pl.col("flag")).sum()
)
```
Today, cudf-polars generally has to lower this kind of expression by materializing an intermediate filtered/masked column, for example:
```python
where(flag, value, null).sum()
```
That only works when null is a valid identity for the aggregation semantics, and it can be more expensive than applying the predicate directly inside the aggregation.
## Additional motivation: SQL FILTER and null-observing aggregations
This is also needed for SQL-style `FILTER (WHERE ...)` aggregation semantics, as described in #23233 . The predicate must represent row selection, not null injection. Rewriting filtered-out rows to null can produce incorrect results for null-observing/null-preserving aggregations such as `collect_list` / `array_agg`, `collect_set`, or `map_agg`, because masked-out rows may become visible null elements instead of being excluded entirely.
## Desired behavior
libcudf aggregation APIs should be able to accept an optional per-aggregation boolean predicate/mask and aggregate only rows where the predicate is true.
The predicate/mask should be applied as row selection independently of null validity: masked-out rows are excluded from the aggregation input, while real nulls in selected rows are preserved and handled according to the aggregation’s normal null semantics.
This would make it possible to lower filtered aggregations directly and avoid aggregation-specific null-identity rewrites.
Contributor guide
Research direction
Start by reading the libcudf aggregation APIs and the linked SQL FILTER discussion in #23233 to understand the required semantics. The issue does not name specific files or tests, so locate the aggregation entry points and their tests before estimating implementation scope. Done means aggregations accept an optional boolean predicate, exclude masked-out rows, and preserve real nulls according to each aggregation’s existing semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100