NVIDIA / NVIDIA/cudf

[FEA] Properly interface with `dask` `object` -> `arrow[string]` conversions

Open
#14,915 0 comments 1 reaction 0 assignees View on GitHub
cudf.pandas dask feature request Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
`dask` has a behavioral change when `pandas-2.x` & `pyarrow` are installed. Some APIs try switching `object` type columns to `arrow[string]` for efficiencies arrow strings provide over object types in pandas. This causes a lot of pytest failures in `dask_cudf`, that is currently being prevented by configured a `dask.config`: `dataframe.convert-string` to `False`. However, this only happens properly for `dask.DataFrame`'s and not for `dask_cudf.DataFrame`'s.

**Describe the solution you'd like**
We need to update `cudf` & `dask` upstream to get this support for `dask_cudf.DataFrame` as well when the device object is converted to host objects.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.