[FEA] Allow groupby scan aggregations to return listified results
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
To match the way scan aggregation results in groupby operations are returned in pandas, libcudf returns scan-based aggregations in the same shape as the input table (these are then optionally reordered to mimic pandas order).
For the cudf-polars executor, it would be useful to also have a mode where the result of a scan aggregation is collected, group-wise, into a list column. This would mean that both scan-like and reduction-like groupby aggregations always produce an output table with a number of rows equal to the number of unique group keys.
**Describe the solution you'd like**
For scan-only aggregations, this is relatively easy to achieve by taking the sorted grouped result and calling `make_lists_column` with the group offsets. When mixing scan and hash-based aggregations it is tricker (since those would spit things out in a different order and would then need a join). Ideally one the scan aggs have a "collect as list" option, then one would be able to do scan and reduce- aggs in the same call on the sorted table.
**Describe alternatives you've considered**
I can post-process the result (and then do a join if I have any hash-based aggs in addition).
**Additional context**
Right now, polars guarantees that although the order of groups in the result is implementation dependent, within a group, the rows show up in original dataframe order. I think this is also guaranteed by libcudf, since the sort-by-key before the aggregations is stable.
Contributor guide
Assessment
This issue has not been assessed yet.