NVIDIA / NVIDIA/cudf

Partition distribution across GPUs when using dask_cudf

Open
#18,004 1 comment 0 reactions 1 assignee Claimed by @TomAugspurger View on GitHub
dask Python question
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

When reading data from a CSV file, it is necessary to distribute the partitions across different GPUs. However, an error occurs when calling the to_device method.
df = dask_cudf.read_csv('./*.csv')
df = df.map_partitions(lambda part, i: part.to_device(i % 2))
KeyError: 'to_device'
During handling of the above exception, another exception occurred:
AttributeError: DataFrame object has no attribute to_device
Could you please help me understand what the error might be? Or is it necessary to use a different approach?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.