NVIDIA / NVIDIA/cudf

[FEA] Async mode for cudf.series operations

Open
#13,087 5 comments 0 reactions 0 assignees View on GitHub
0 - Backlog feature request libcudf Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

We get wide dataframes in situations like machine learning (easily 1-5K cols) and genomics (10K+ cols), and while there is some speedup from cudf (say 2-3X), it'd be easy to get to the 10X+ level with much higher GPU utilization if we could spawn concurrent tasks for each column . Getting this all the way to the df level seems tricky, but async primitives at the column level would get us far.

One Python-native idea is doing via `async/await`, when one cudf operation is getting scheduled, allocated, & run, we can be scheduling the next, and ideally, cudf can run them independently . It smoothed out 2-3 years ago in python + javascript as a popular native choice, and has since been a lot more popular in pydata, e.g., langchain just rewrote to support async versions of all methods. Ex: https://trends.google.com/trends/explore?date=all&q=async%20await&hl=en . Separately, there's heightened value for pydata dashboarding scenarios like plotly, streamlit, etc as these ecosystem increasingly build for async io underneath as well.

(Another idea with precedent is a lazy mode similar to haskell or dask, discussed below as well)

**Describe the solution you'd like**

I'd like to be do something like:

```python

async def f(s: cudf.Series) -> cudf.Series:
# async mode for core series operations lets other f() calls proceed while this runs
s2 = await s.stra.hex_to_int('AABBCC')

# math can be clean and enable the same
# if we're super clever, this may even unlock query plan optimizations like fusion in the future
async with cudf.async.binop_mode:
s3_a = s2 + 1 / 3
s3 = await s3_a

return s3

cols2 = await async.gather([ f(df[col]) for col in df ])
```

**Describe alternatives you've considered**

1. Use existing abstractions

In theory we can setup threads or multiple dask workers, but (1) both are super awkward, (2) underneath, cudf will not do concurrent jobs

2. Lazy cudf

Another thought is to create a lazy mode for cudf. This has precedent with Haskell, and in modern pydata land, more so with polars. Dask does this too, and we'd use it if that can work, but it's awkward -- I haven't used, but polars sounds to be more friendly in practice:

```python

def f(s: cudf.Series) -> cudf.Series:
# explicitly lazy ops
s2 = s.str_lazy.hex_to_int('AABBCC')

# binops know they're lazy
s3 = s2 + 1 / 3

return s3

# force with async friendliness
cols2 = await cudf_client.compute_async([ f(df[col]) for col in df ])
```

Underneath, cudf can reinvent async/io, dask, or whatever

**Additional context**

Slack thread: https://rapids-goai.slack.com/archives/C5E06F4DC/p1680710488795869

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.