[QST] Respecting pandas options that affect default behaviors
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**What is your question?**
How should pandas global options interact with cudf.pandas? These can change the behavior/result of pandas operations _without_ changing the call being made.
For example, pandas has a `use_inf_as_na` option that makes inf behave like nulls. Should the pandas-path be forced if cudf.pandas detects a global option that isn't supported?
```python
# with %load_ext cudf.pandas
In [3]: ser = pd.Series([np.inf, np.nan])
In [4]: ser
Out[4]:
0 Inf
1
dtype: float64
# Incorrect
In [5]: ser.dropna()
Out[5]:
0 inf
dtype: float64
In [6]: pd.options.mode.use_inf_as_na = True
In [7]: ser
Out[7]:
0 Inf
1
dtype: float64
In [8]: ser.dropna()
Out[8]:
0 inf
dtype: float64
# if we try setting this option in a non-cudf supported way we get a different (incorrect) answer
In [10]: with pd.option_context("use_inf_as_na", True):
...: print(ser.dropna())
...:
0 NaN
dtype: float64
```
The pandas result is:
```
>>> import pandas as pd; import numpy as np
>>> ser = pd.Series([np.inf, np.nan])
>>> pd.options.mode.use_inf_as_na = True
>>> ser.dropna()
Series([], dtype: float64)
```
Contributor guide
Assessment
This issue has not been assessed yet.