[BUG] Passing `fill_value` in `groupby.shift` with a mismatching dtype must result in error
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Describe the bug**
When `fill_values`'s dtype actually is mismatching with a column's dtype, we should raise an error as we don't support mixed types(`'object'`) like in pandas. Pandas is returning a mixed `object` dtypes when there is a mismatch of column dtype & `fill_value` dtype in `1.4.x`
**Steps/Code to reproduce bug**
```python
>>> import pandas as pd
>>> import pandas as pd
>>> s = pd.DataFrame()
>>> s['a'] = [1, 2, 3, 4, 5]
>>> s['b'] = pd.Series([10, 11, 12, 13, 14], dtype='datetime64[ns]')
>>> s.groupby('a').shift(1500, fill_value=10)
b
0 10
1 10
2 10
3 10
4 10
>>> s.groupby('a').shift(1500, fill_value=10).dtypes
b object
dtype: object
>>> import cudf
>>> gs = cudf.from_pandas(s)
>>> gs.groupby('a').shift(1500, fill_value=10)
b
0 1970-01-01 00:00:00.000000010
1 1970-01-01 00:00:00.000000010
2 1970-01-01 00:00:00.000000010
3 1970-01-01 00:00:00.000000010
4 1970-01-01 00:00:00.000000010
>>> gs.groupby('a').shift(1500, fill_value=10).dtypes
b datetime64[ns]
dtype: object
```
**Expected behavior**
Few possible solutions:
1. Raise error that mixed types are not possible like we do in other places of code-base.
2. If we do 1, users will want a way fill values based on dtypes or column names. So expanding `fill_value` to be a dictionary of fill values that can be used to fill values based on dtype like: {"str": "abc", "int64": 10, ....} or based on columns like: {"a":"abc", "b": 10, ...}
In a way, this is both a bug & feature request.
**Environment overview (please complete the following information)**
- Environment location: [Bare-metal]
- Method of cuDF install: [from source]
Contributor guide
Assessment
This issue has not been assessed yet.