NVIDIA / NVIDIA/cudf

[FEA] Support indicator=True in cudf.DataFrame.merge

Open
#13,581 1 comment 0 reactions 0 assignees View on GitHub
0 - Backlog feature request improvement Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
I wish dask_cudf dataframes would support the `indicator=True` option in the merge function

**Describe the solution you'd like**
In dask dataframe the code:
```python
import pandas as pd
import dask.dataframe as dd

dfa = pd.DataFrame({"id":[1,2,2,4], "a":["a","b","c","d"]})
dfb = pd.DataFrame({"id":[2,3,3,4], "b":["e","f","g","h"]})
ddfa = dd.from_pandas(dfa, npartitions=1)
ddfb = dd.from_pandas(dfb, npartitions=1)

ddf = ddfa.merge(ddfb, on="id", how="outer", indicator=True)
print(ddf.compute())
```
returns:
```
id a b _merge
0 1 a NaN left_only
1 2 b e both
2 2 c e both
3 4 d h both
4 3 NaN f right_only
5 3 NaN g right_only
```

but in dask_cudf:
```python
import cudf
import dask_cudf as dd

dfa = cudf.DataFrame({"id":[1,2,2,4], "a":["a","b","c","d"]})
dfb = cudf.DataFrame({"id":[2,3,3,4], "b":["e","f","g","h"]})
ddfa = dd.from_cudf(dfa, npartitions=1)
ddfb = dd.from_cudf(dfb, npartitions=1)

ddf = ddfa.merge(ddfb, on="id", how="outer", indicator=True)
print(ddf.compute())
```
this throws a `NotImplementedError`:
```
Traceback (most recent call last):
File "/raid/cjarrett/dask-sql/min.py", line 13, in
ddf = ddfa.merge(ddfb, on="id", how="outer", indicator=True)
File "/raid/cjarrett/mambaforge/envs/dsql-6-6/lib/python3.10/site-packages/nvtx/nvtx.py", line 101, in inner
result = func(*args, **kwargs)
File "/raid/cjarrett/mambaforge/envs/dsql-6-6/lib/python3.10/site-packages/dask_cudf/core.py", line 121, in merge
return super().merge(
File "/raid/cjarrett/mambaforge/envs/dsql-6-6/lib/python3.10/site-packages/dask/dataframe/core.py", line 5644, in merge
return merge(
File "/raid/cjarrett/mambaforge/envs/dsql-6-6/lib/python3.10/site-packages/dask/dataframe/multi.py", line 724, in merge
return hash_join(
File "/raid/cjarrett/mambaforge/envs/dsql-6-6/lib/python3.10/site-packages/dask/dataframe/multi.py", line 398, in hash_join
meta = _lhs_meta.merge(_rhs_meta, **kwargs)
File "/raid/cjarrett/mambaforge/envs/dsql-6-6/lib/python3.10/site-packages/nvtx/nvtx.py", line 101, in inner
result = func(*args, **kwargs)
File "/raid/cjarrett/mambaforge/envs/dsql-6-6/lib/python3.10/site-packages/cudf/core/dataframe.py", line 3969, in merge
raise NotImplementedError(
NotImplementedError: Only indicator=False is currently supported
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.