[FEA] Support aggregations/scans on lists via groupby
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
This is related to https://github.com/rapidsai/cudf/issues/10408.
It's possible to use a combination of `explode()` and `groupby()` to support common aggregations on `list` columns:
```python
In [23]: df
Out[23]:
a
0 [1, 3, 2]
1 [1, 4]
2 [9, 0, -1]
In [24]: df.explode('a').groupby(level=0).max()
Out[24]:
a
0 3
1 4
2 9
```
Scans can similarly be computed via an additional call to `groupby-collect()`:
```python
In [33]: df.explode('a').groupby(level=0).cummax().groupby(level=0).collect()
Out[33]:
a
0 [1, 3, 3]
1 [1, 4]
2 [9, 9, 9]
```
When an aggregation/scan supported by groupby, like `.max()` or `.min()` is called on a list column, we could transparently use this combination of explode + groupby to support that operation. That could look something like:
```python
>>> df
a
0 [1, 3, 2]
1 [1, 4]
2 [9, 0, -1]
>>> df['a'].list.max()
0 3
1 4
2 9
Name: a, dtype: int64
>>> df['a'].list.cummax()
0 [1, 3, 3]
1 [1, 4]
2 [9, 9, 9]
Name: a, dtype: list
```
Contributor guide
Assessment
This issue has not been assessed yet.