NVIDIA / NVIDIA/cudf

[BUG] Some custom dask aggregations fail with dask_cudf dataframes

Open
#11,515 4 comments 0 reactions 0 assignees View on GitHub
0 - Backlog feature request Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Describe the bug**
Some dask custom aggregations (ex: a custom sum of squares aggregation) fail with dask_cudf.

**Steps/Code to reproduce bug**
```
import cudf
import dask_cudf
import dask.dataframe as dd

df = cudf.DataFrame({"a":[1,2,3], "b":[1,1,2]})
ddf = dask_cudf.from_cudf(df, npartitions=1)

sum_of_squares = dd.Aggregation(
name='sum_of_squares',
chunk=lambda s: s.agg(lambda x: (x**2).sum()),
agg=lambda s0: s0.sum()
)

ddf.groupby("b").agg(sum_of_squares).compute()
```
returns:
```
TypeError: unsupported operand type(s) for ** or pow(): 'type' and 'int'
```
however when running with a pandas backed dask dataframe:
```
ddf.to_dask_dataframe().groupby("b").agg(sum_of_squares).compute()
```
the expected result is returned:
```
a
b
1 5
2 9

```

cc: @randerzander

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.