NVIDIA / NVIDIA/cudf

[BUG] Expose groupby rank via agg interface

Open
#12,157 0 comments 0 reactions 0 assignees View on GitHub
bug Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

In pandas, I can compute groupby rank via the agg interface. In cuDF, I cannot.

This came up while looking into SQL window function coverage.

```python
import cudf

df = cudf.DataFrame({
"a": [0, 0, 0, 1, 1, 1],
"val": [5, 3, 7, 3, 9, 4],
})
print(df.groupby("a").val.rank())
print(df.to_pandas().groupby("a").agg({"val":["rank"]}))
df.groupby("a").agg({"val":["rank"]})
0 2.0
1 1.0
2 3.0
3 1.0
4 3.0
5 2.0
Name: val, dtype: float64
val
rank
0 2.0
1 1.0
2 3.0
3 1.0
4 3.0
5 2.0

---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Input In [55], in ()
7 print(df.groupby("a").val.rank())
8 print(df.to_pandas().groupby("a").agg({"val":["rank"]}))
----> 9 df.groupby("a").agg({"val":["rank"]})

File ~/miniconda3/envs/rapids-22.12/lib/python3.9/contextlib.py:79, in ContextDecorator.__call__..inner(*args, **kwds)
76 @wraps(func)
77 def inner(*args, **kwds):
78 with self._recreate_cm():
---> 79 return func(*args, **kwds)

File ~/miniconda3/envs/rapids-22.12/lib/python3.9/site-packages/cudf/core/groupby/groupby.py:460, in GroupBy.agg(self, func)
451 column_names, columns, normalized_aggs = self._normalize_aggs(func)
453 # Note: When there are no key columns, the below produces
454 # a Float64Index, while Pandas returns an Int64Index
455 # (GH: 6945)
456 (
457 result_columns,
458 grouped_key_cols,
459 included_aggregations,
--> 460 ) = self._groupby.aggregate(columns, normalized_aggs)
462 result_index = self.grouping.keys._from_columns_like_self(
463 grouped_key_cols,
464 )
466 multilevel = _is_multi_agg(func)

File groupby.pyx:302, in cudf._lib.groupby.GroupBy.aggregate()

File groupby.pyx:250, in cudf._lib.groupby.GroupBy.scan_internal()

File aggregation.pyx:897, in cudf._lib.aggregation.make_groupby_scan_aggregation()

File aggregation.pyx:573, in cudf._lib.aggregation.GroupbyScanAggregation.rank()

TypeError: rank() takes exactly 4 positional arguments (0 given)
````

```
conda list | grep "rapids\|pandas"
# packages in environment at /home/nicholasb/miniconda3/envs/rapids-22.12:
cubinlinker 0.2.0 py39h11215e4_1 rapidsai-nightly
cucim 22.12.00a221115 cuda_11_py39_gf085f29_10 rapidsai-nightly
cudf 22.12.00a221115 cuda_11_py39_gbae9e39ffa_261 rapidsai-nightly
cudf_kafka 22.12.00a221115 py39_gbae9e39ffa_261 rapidsai-nightly
cugraph 22.12.00a221115 cuda11_py39_g85c447b0_108 rapidsai-nightly
cuml 22.12.00a221115 cuda11_py39_g65008974f_41 rapidsai-nightly
cusignal 22.12.00a221115 py39_g5265ccb_7 rapidsai-nightly
cuspatial 22.12.00a221115 py39_g337a92a_59 rapidsai-nightly
custreamz 22.12.00a221115 py39_gbae9e39ffa_261 rapidsai-nightly
cuxfilter 22.12.00a221115 py39_gbeb436a_4 rapidsai-nightly
dask-cuda 22.12.00a221115 py39_gb3ed902_27 rapidsai-nightly
dask-cudf 22.12.00a221115 cuda_11_py39_gbae9e39ffa_261 rapidsai-nightly
datashader 0.13.1a py_0 rapidsai-nightly
geopandas 0.12.1 pyhd8ed1ab_1 conda-forge
geopandas-base 0.12.1 pyha770c72_1 conda-forge
libcucim 22.12.00a221115 cuda11_gf085f29_10 rapidsai-nightly
libcudf 22.12.00a221115 cuda11_gbae9e39ffa_261 rapidsai-nightly
libcudf_kafka 22.12.00a221115 gbae9e39ffa_261 rapidsai-nightly
libcugraph 22.12.00a221115 cuda11_g605e2b53_109 rapidsai-nightly
libcugraph_etl 22.12.00a221115 cuda11_g605e2b53_109 rapidsai-nightly
libcugraphops 22.12.00a221115 cuda11_gbbf866a_20 rapidsai-nightly
libcuml 22.12.00a221115 cuda11_g65008974f_41 rapidsai-nightly
libcumlprims 22.12.00a221010 cuda11_geaadb5e_2 rapidsai-nightly
libcuspatial 22.12.00a221115 cuda11_g337a92a_59 rapidsai-nightly
libraft-distance 22.12.00a221115 cuda11_g355f693_114 rapidsai-nightly
libraft-headers 22.12.00a221115 cuda11_g355f693_114 rapidsai-nightly
libraft-nn 22.12.00a221115 cuda11_g355f693_114 rapidsai-nightly
librmm 22.12.00a221115 cuda11_g4da70d53_54 rapidsai-nightly
libxgboost 1.6.2dev.rapidsai22.12 cuda_11_0 rapidsai-nightly
pandas 1.5.1 py39h4661b88_1 conda-forge
ptxcompiler 0.7.0 cuda_11_py39_gb86c990_1 rapidsai-nightly
py-xgboost 1.6.2dev.rapidsai22.12 cuda_11_py39_0 rapidsai-nightly
pylibcugraph 22.12.00a221115 cuda11_py39_g85c447b0_108 rapidsai-nightly
pylibraft 22.12.00a221115 cuda11_py39_g355f693_114 rapidsai-nightly
raft-dask 22.12.00a221115 cuda11_py39_g355f693_114 rapidsai-nightly
rapids 22.12.00a221115 cuda11_py39_g2d3c297_34 rapidsai-nightly
rapids-xgboost 22.12.00a221115 cuda11_py39_g2d3c297_34 rapidsai-nightly
rmm 22.12.00a221115 cuda11_py39_g4da70d53_54 rapidsai-nightly
strings_udf 22.12.00a221115 cuda_11_py39_gbae9e39ffa_261 rapidsai-nightly
ucx-proc 1.0.0 gpu rapidsai-nightly
ucx-py 0.29.00a221114 py39_g5c7f2a3_19 rapidsai-nightly
xgboost 1.6.2dev.rapidsai22.12 cuda_11_py39_0 rapidsai-nightly
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.