NVIDIA / NVIDIA/cudf

[FEA] Share PDSH benchmarking CLI between cudf.pandas and cudf_polars

Open
#20,183 2 comments 0 reactions 0 assignees View on GitHub
feature request Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

https://github.com/rapidsai/cudf/pull/20164 essentially copies and prunes unneeded functionality from `cudf_polars/cudf_polars/experimental/benchmarks/utils.py` to run PDSH benchmarks with cudf.pandas. In the long term, I believe `cudf_polars/cudf_polars/experimental/benchmarks/utils.py` could be used run the pandas-implemented benchmarks with the following generalizations:

1. Make `PackageVersion` more configurable https://github.com/rapidsai/cudf/blob/6cd2a4f80f443bfef112655a5829286a07fc46c9/python/cudf_polars/cudf_polars/experimental/benchmarks/utils.py#L115-L118
2. `run_polars` and `execute_query` are almost fully generalized, but `run_polars` could further generalize validation to a CPU library, implementation specific logging (e.g. query plan), implementation specific tracing.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.