[FEA] Add a `cast_dtype` argument to `partition_by_hash`
Open
dask
feature request
libcudf
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
Dask CUDA's explicit-comms shuffle algorithm uses the `partition_by_hash` API to efficiently partition a DataFrame by hashing a subset of its columns. However, after https://github.com/dask/dask/pull/10705, the output of `partitioning_index` is often inconsistent with that of `partition_by_hash` for numerical data (due to dtype casting).
In order to use `partition_by_hash` in dask-CUDA, something similar to `cast_dtype` must be added.
Contributor guide
Assessment
This issue has not been assessed yet.