NVIDIA / NVIDIA/cudf

[BUG] groupby.fillna(method="ffill") doesn't work for NaN

Open
#10,365 5 comments 0 reactions 0 assignees View on GitHub
bug Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Describe the bug**
groupby.fillna(method="ffill") doesn't work for NaN. bfill also has same issue.

**Steps/Code to reproduce bug**
```
In [3]: df = cudf.DataFrame(
...: [
...: ["A", 1],
...: ["A", np.nan],
...: ["B", 0],
...: ["B", None],
...: ],
...: columns=["x", "y"],
...: )
...:

In [4]: df
Out[4]:
x y
0 A 1
1 A
2 B 0
3 B
```
fillna works for < NA >
```
In [5]: df.groupby("x")["y"].fillna(method="ffill")
...:
Out[5]:
0 1
1 1
2 0
3 0
Name: y, dtype: int64
```
But it doesn't work for NaN
```
In [6]: df.loc[df["y"].isnull(), "y"] = np.nan
...: df
Out[6]:
x y
0 A 1.0
1 A NaN
2 B 0.0
3 B NaN

In [7]: df.groupby("x")["y"].fillna(method="ffill")
...:
Out[7]:
0 1.0
1 NaN
2 0.0
3 NaN
Name: y, dtype: float64
```
We need to replace NaN with < NA > using None
```
In [8]: df.loc[df["y"].isnull(), "y"] = None
...: df
Out[8]:
x y
0 A 1.0
1 A
2 B 0.0
3 B

In [9]: df.groupby("x")["y"].fillna(method="ffill")
...:
Out[9]:
0 1.0
1 1.0
2 0.0
3 0.0
Name: y, dtype: float64
```

**Environment overview (please complete the following information)**
cudf version is "22.02.00"

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.