[FEA]Coalesce(), find the first non-null value, no equivalent function in RAPIDS
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
I want to create a new column, which is the first non-null value of several columns, I used the function [coalesce()](https://www.rdocumentation.org/packages/dplyr/versions/0.7.8/topics/coalesce) in dplyr to achieve that by simply run ` data %>% mutate(d = coalesce(a, b, c))`, in Dask CPU, I run below code to achieve that
```
d = {'a': [1, 2, 3, None, 5, 1, None, None, None, None, 3, None, 5, 6, 7], \
'b': [1, 2, 3, 200, 5, 1, 200, None, 400, None, 3, None, 5, 6, 7], \
'c': [1, 2, 3, None, 5, 1, 300, 300, None, 600, 3, 300, 5, 6, 7]}
df2 = pd.DataFrame(d)
ddf2 = dd.from_pandas(df2,npartitions=50)
ddf2['d'] = ddf2['a'].copy().fillna(ddf2['b']).fillna(ddf2['c'])
ddf2.compute()
```
However, in dask cuDF, when I am trying to use the fillna() to acheive my intent, I got the error message
```
d = {'id': ['a', 'a','a','a','a','b','b','b','b','b','c','c','c','c','c'], \
'time': ['1', '2', '4', '3', '5', '1', '2', '3', '4', '5','1', '2', '3', '4', '5'], \
'a': [1, 2, 3, None, 5, 1, None, None, None, None, 3, None, 5, 6, 7], \
'b': [1, 2, 3, 200, 5, 1, 200, None, 400, None, 3, None, 5, 6, 7], \
'c': [1, 2, 3, None, 5, 1, 300, 300, None, 600, 3, 300, 5, 6, 7]}
df = pd.DataFrame(data=d)
gdf = cudf.DataFrame.from_pandas(df)
ddf = dask_cudf.from_cudf(gdf, npartitions=4)
ddf['d'] = ddf['a'].copy().fillna(ddf['b']).fillna(ddf['c'])
ddf.compute()
```
error message: `RuntimeError: cuDF failure at: /opt/conda/envs/rapids/conda-bld/libcudf_1598487636199/work/cpp/src/replace/replace.cu:804: Column size mismatch`
So I am wondering if there are functions that allows me to get the first non-null value from several columns in cuDF or if we can add that.
Contributor guide
Assessment
This issue has not been assessed yet.