[BUG] Incorrect `NaN` ignoring in GroupBy JIT Reductions
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Describe the bug**
Pandas series and dataframe reductions default to `skipna=True`, our JIT operators don't fully support this behavior.
**Steps/Code to reproduce bug**
```python
import cudf
df = cudf.DataFrame({
'key1': [0,0],
'val1':[1, float('nan')]
}, nan_as_null=False)
def func(df):
return df['val1'].sum()
expect = df.to_pandas().groupby('key1').apply(func)
got = df.groupby('key1').apply(func, engine='jit')
print(expect)
print(got)
```
```
key1
0 1.0
dtype: float64
key1
0 NaN
dtype: float64
```
**Expected behavior**
`NaN` values should be ignored by default and the same numerical value should be returned from cuDF as pandas.
**Environment overview (please complete the following information)**
- Environment location: Bare-metal
- Method of cuDF install: Source
**Additional context**
The current operators have _some_ `NaN` handling, but it's mostly to cover cases where the entire sequence of values is `NaN` or for edge cases in some operators where we'd end up dividing by zero. However general `NaN` _skipping_ is not implemented leading to the buggy behavior.
Contributor guide
Assessment
This issue has not been assessed yet.