NVIDIA / NVIDIA/cudf

[QST] Respecting pandas options that affect default behaviors

Open
#14,403 1 comment 0 reactions 0 assignees View on GitHub
cudf.pandas Python question
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**What is your question?**
How should pandas global options interact with cudf.pandas? These can change the behavior/result of pandas operations _without_ changing the call being made.

For example, pandas has a `use_inf_as_na` option that makes inf behave like nulls. Should the pandas-path be forced if cudf.pandas detects a global option that isn't supported?

```python
# with %load_ext cudf.pandas

In [3]: ser = pd.Series([np.inf, np.nan])

In [4]: ser
Out[4]:
0 Inf
1
dtype: float64

# Incorrect
In [5]: ser.dropna()
Out[5]:
0 inf
dtype: float64

In [6]: pd.options.mode.use_inf_as_na = True

In [7]: ser
Out[7]:
0 Inf
1
dtype: float64

In [8]: ser.dropna()
Out[8]:
0 inf
dtype: float64

# if we try setting this option in a non-cudf supported way we get a different (incorrect) answer
In [10]: with pd.option_context("use_inf_as_na", True):
...: print(ser.dropna())
...:
0 NaN
dtype: float64
```

The pandas result is:

```
>>> import pandas as pd; import numpy as np
>>> ser = pd.Series([np.inf, np.nan])
>>> pd.options.mode.use_inf_as_na = True
>>> ser.dropna()
Series([], dtype: float64)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.