NVIDIA / NVIDIA/cudf

[BUG]Groupby agg with np.nan in aggregating column leads to incorrect results

Open
#6,810 6 comments 0 reactions 0 assignees View on GitHub
bug Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Describe the bug**

If we have nulls in the column to run aggregate on we, seem to get incorrect results if there are `np.nan` in the column we are aggregating on.

**Steps/Code to reproduce bug**
```python

>>> import numpy as np
>>> import cudf
>>> df = cudf.DataFrame({'key':[0,0,0],'val':[None,1.1,None]})
>>> df['val']=df.val.fillna(np.nan)
>>> df.groupby(by='key').val.sum()
key
0 NaN
Name: val, dtype: float64
```

**Expected behavior**

```python
>>> df.to_pandas().groupby(by='key').val.sum()
key
0 1.1
Name: val, dtype: float64
```

**Environment details**
```python
cudf 0.17.0a201119 cuda_10.2_py37_g1a80df96c4_285 rapidsai-nightly
libcudf 0.17.0a201119 cuda10.2_g1a80df96c4_285 rapidsai-nightly
```

**Additional context**
As a workaround below does the trick.

```python
df.nans_to_nulls().groupby(by='key').agg({'val':['sum']})
```

CC: @beckernick

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.