[DOC]: Avoid UserWarning in 10 minute intro
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
## Report incorrect documentation
**Location of incorrect documentation**
https://docs.rapids.ai/api/cudf/nightly/user_guide/10min/#persisting-data
**Describe the problems or issues found in the documentation**
That includes a UserWarning from dask.dataframe because we create a large in-memory dataframe and send it to the worker, rather than creating / loading data on the worker:
```
ddf2 = ddf2.persist()
ddf2
```
which prints
```
/opt/conda/envs/docs/lib/python3.13/site-packages/distributed/client.py:3363: UserWarning: Sending large graph of size 152.59 MiB.
This may cause some slowdown.
Consider loading the data with Dask directly
or using futures or delayed objects to embed the data into the graph without repetition.
See also https://docs.dask.org/en/stable/best-practices.html#load-data-with-dask for more information.
warnings.warn(
```
**Steps taken to verify documentation is incorrect**
List any steps you have taken:
**Suggested fix for documentation**
We can use `dask.dataframe.from_map` to create the data on the worker. We need a helper function to get the offset right:
```python
nrows = 10000000
npartitions = 16
nrows_per_partition = nrows // npartitions
offsets = [i * nrows_per_partition for i in range(npartitions)]
def make_partition(offset, nrows_per_partition):
values = cp.arange(nrows_per_partition) + offset
return cudf.DataFrame({"a": values, "b": values}, index=values)
ddf2 = dd.from_map(make_partition, offsets, nrows_per_partition=nrows_per_partition)
ddf2["c"] = ddf2["a"] + 5
```
There's a second one later that uses more partitions. Repeat with a different `npartitions` / `nrows_per_partition`.
Contributor guide
Assessment
This issue has not been assessed yet.