NVIDIA / NVIDIA/cudf

[BUG] Error with dask-expr using categorical dtype

Open
#17,415 5 comments 0 reactions 0 assignees View on GitHub
bug Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

This is the same issue as https://github.com/rapidsai/dask-cuda/issues/1408 . Cross-posting here as it's more related to cuDF instead of `dask-cuda`.

The following snippet works with `DASK_DATAFRAME__QUERY_PLANNING=FALSE` but fails with `dask-expr`.

``` python
import dask
import dask.dataframe as dd
import dask_cuda
from dask_cuda import LocalCUDACluster
from distributed import Client

def main(client):
print(dask_cuda.__version__)
print(dask.__version__)
df = dd.from_dict({"qid": [1, 2, 1, 0, 2]}, npartitions=3)
df.qid.astype("category").cat.as_known().compute()

if __name__ == "__main__":
with LocalCUDACluster() as cluster:
with Client(cluster) as client:
with dask.config.set(
{"array.backend": "cupy", "dataframe.backend": "cudf"}
):
main(client)
```

```
Traceback (most recent call last):
File "/home/jiamingy/workspace/xgboost_dev/XGBoostUtils/dask-issues/as-cat.py", line 20, in
main(client)
File "/home/jiamingy/workspace/xgboost_dev/XGBoostUtils/dask-issues/as-cat.py", line 11, in main
df.qid.astype("category").cat.as_known().compute()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jiamingy/.anaconda/envs/xgboost_dev_125/lib/python3.11/site-packages/dask_expr/_categorical.py", line 83, in as_known
return self.set_categories(categories.values)
^^^^^^^^^^^^^^^^^
File "/home/jiamingy/.anaconda/envs/xgboost_dev_125/lib/python3.11/site-packages/cudf/core/index.py", line 1636, in values
return self._column.values
^^^^^^^^^^^^^^^^^^^
File "/home/jiamingy/.anaconda/envs/xgboost_dev_125/lib/python3.11/site-packages/cudf/core/column/string.py", line 5873, in values
raise TypeError("String Arrays is not yet implemented in cudf")
TypeError: String Arrays is not yet implemented in cudf
```

**Environment overview (please complete the following information)**
- Environment location: Bare-metal
- Method of cuDF install: conda

- dask-cuda: 24.12.00a12
- dask: 2024.10.0
- Python: 3.11.10

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.