NVIDIA / NVIDIA/cudf

[FEA] Update cudf.pandas Parquet reading according to cudf-polars results

Open
#17,225 0 comments 0 reactions 0 assignees View on GitHub
feature request Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

Based on our findings in https://github.com/rapidsai/cudf/pull/16944 we may want to update cudf.pandas to parametrize the chunked parquet reading accordingly, in particular by setting a new pass read limit. The exact value may need to be different based on 1) cudf.pandas generally targeting smaller GPUs and 2) cudf.pandas using UVM, so we may have to do some extra research and testing there.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.