[FEA] cudf-polars: keep collect() results or intermediate persist() values on-device, especially for single-GPU workflows
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
We're adding a polars, polars-cudf backend for the GFQL's property graph query system in PyGraphistry, including the popular Cypher language. We're seeing encouraging results using cudf-polars to eliminate per-op overhead via a collect-once lazy plan.
The remaining friction is that there's no supported way to keep the result of a query on the GPU. Today we can do device-to-device handoffs that stay resident from regular cudf -> polars-cudf, but LazyFrame.collect() and .sink_*() both move the result off-device (D→H), so we cannot chain polars-cudf -> cudf nor polars-cudf -> polars-cudf. This hurts us in two distinct ways:
1. Benchmarking accuracy. Because every collect() pays a D→H copy of the result, our GPU timings include a host-transfer tax that isn't part of the actual compute. We can't cleanly isolate GPU compute cost from the transfer cost, which makes it hard to fairly compare cudf-polars against the native cuDF engine (which stays device-resident) for the same query.
2. True on-device chaining (the real goal). Our workload is iterative/chained — a graph already resident on the GPU, queried repeatedly (e.g. a crossfilter / multi-hop traversal loop). We'd like to keep the device result resident and feed it into the next query without a round-trip to host, i.e. a D→D flow that pays only command + on-device copy cost, not D→H→D.
**Describe the solution you'd like**
A supported, public way to keep a query result device-resident after execution — conceptually similar to dask-cudf's .persist() and .compute() — such that:
- the materialized result stays on the GPU (no implicit D→H on collect/sink), and
- that device-resident result can be used as the input to a subsequent polars/cudf-polars query without a host round-trip.
We understand from discussion with the cudf-polars team that the in-progress polars execute() work is the principled mechanism that will enable this, including the D→D chaining case. This issue is to register our use case and be notified when that lands.
**Describe alternatives you've considered**
- limit(1) / truncating the output in benchmarks to shrink the D→H cost — rejected as risky, since the polars optimizer may prune real work we intended to measure.
- The unsupported in-memory recipe Lawrence shared (build a Translator over q._ldf.visit() with pl.GPUEngine(executor="in-memory"), translate_ir(), then ir.evaluate(...) under set_memory_resource(...) to get a device_df wrapping a pylibcudf Table). This defers D→H and is usable for benchmarking, but: it's private API ("caveat emptor, if it breaks you keep both pieces"), it only works with the in-memory executor (not the default streaming executor), the device_df exposes few methods, and it cannot be fed into a subsequent polars query (only .to_polars() back to host) — so it does not enable the D→D chaining we actually want.
**Additional context**
- Single-node, single-GPU is our primary case; multi-GPU is a later concern.
- The lack of D->D, in practice, eliminates utility for many of our non-trivial use cases
- Our two needs are separable: (a) exclude D→H from benchmark measurements (the in-memory hack covers this today, unsupported), and (b) genuine device-resident result reuse across chained queries (needs the upcoming execute() work).
- Per the team, the streaming executor does not support the current hack, and execute() is the intended path for the D→D reuse case.
- Happy to share a concrete reproducer of our chained/iterative GPU query pattern if useful.
Contributor guide
Assessment
This issue has not been assessed yet.