[BUG] dataframe reindex NaN
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Describe the bug**
cuDF DataFrame is not reindexed as intented. Using a multiindex, it ends up in NaNs.
**Steps/Code to reproduce bug**
```
data_df = cudf.read_parquet('../input/optiver-realized-volatility-prediction/book_train.parquet/stock_id=0/c439ef22282f412ba39e9137a3fdabac.parquet')
offsets = data_df.groupby(['time_id'], as_index=False).agg({'seconds_in_bucket':'min'}).reset_index(drop=True)
offsets.columns = ['time_id', 'offset']
data_df = cudf.merge(data_df, offsets, on = ['time_id'], how = 'left')
data_df.seconds_in_bucket = data_df.seconds_in_bucket - data_df.offset
# MultiIndex.from_product uses pandas in the background
# That's why we need to transform the data into pd dataframe
data_df = data_df.set_index(['time_id', 'seconds_in_bucket'])
columns = [col for col in data_df.columns.values]
data_df = data_df.reindex(cudf.MultiIndex.from_product([data_df.to_pandas().index.levels[0], np.arange(0,600)], names = ['time_id', 'seconds_in_bucket']), columns=columns).fillna(method='ffill')
data_df = cudf.DataFrame(data_df.reset_index())
```
https://www.kaggle.com/medali1992/optiver-train-dataset?scriptVersionId=68637709
This workaround works:
```
indices = cudf.MultiIndex.from_product([data_df.to_pandas().index.levels[0], np.arange(0,600)], names = ['time_id', 'seconds_in_bucket'])
data_df = cudf.DataFrame().set_index(indices).join(data_df, how="left").fillna(method='ffill').reset_index(drop=True)
```
**Expected behavior**
I expect reindex function to get the correct values instead of NaNs as in pandas.
**Environment overview (please complete the following information)**
Kaggle GPU Docker, RAPIDS 21.06
Contributor guide
Assessment
This issue has not been assessed yet.