[BUG] Error with dask-expr using categorical dtype
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
This is the same issue as https://github.com/rapidsai/dask-cuda/issues/1408 . Cross-posting here as it's more related to cuDF instead of `dask-cuda`.
The following snippet works with `DASK_DATAFRAME__QUERY_PLANNING=FALSE` but fails with `dask-expr`.
``` python
import dask
import dask.dataframe as dd
import dask_cuda
from dask_cuda import LocalCUDACluster
from distributed import Client
def main(client):
print(dask_cuda.__version__)
print(dask.__version__)
df = dd.from_dict({"qid": [1, 2, 1, 0, 2]}, npartitions=3)
df.qid.astype("category").cat.as_known().compute()
if __name__ == "__main__":
with LocalCUDACluster() as cluster:
with Client(cluster) as client:
with dask.config.set(
{"array.backend": "cupy", "dataframe.backend": "cudf"}
):
main(client)
```
```
Traceback (most recent call last):
File "/home/jiamingy/workspace/xgboost_dev/XGBoostUtils/dask-issues/as-cat.py", line 20, in
main(client)
File "/home/jiamingy/workspace/xgboost_dev/XGBoostUtils/dask-issues/as-cat.py", line 11, in main
df.qid.astype("category").cat.as_known().compute()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jiamingy/.anaconda/envs/xgboost_dev_125/lib/python3.11/site-packages/dask_expr/_categorical.py", line 83, in as_known
return self.set_categories(categories.values)
^^^^^^^^^^^^^^^^^
File "/home/jiamingy/.anaconda/envs/xgboost_dev_125/lib/python3.11/site-packages/cudf/core/index.py", line 1636, in values
return self._column.values
^^^^^^^^^^^^^^^^^^^
File "/home/jiamingy/.anaconda/envs/xgboost_dev_125/lib/python3.11/site-packages/cudf/core/column/string.py", line 5873, in values
raise TypeError("String Arrays is not yet implemented in cudf")
TypeError: String Arrays is not yet implemented in cudf
```
**Environment overview (please complete the following information)**
- Environment location: Bare-metal
- Method of cuDF install: conda
- dask-cuda: 24.12.00a12
- dask: 2024.10.0
- Python: 3.11.10
Contributor guide
Assessment
This issue has not been assessed yet.