Enable cudf-polars parquet metadata prefetching by default on-prem
Open
cudf-polars
feature request
Performance
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
https://github.com/rapidsai/cudf/pull/22700 changes cudf-polars execution to add an explicit metadata prefetching stage to query execution, just after lowering and just before executing the actual query nodes.
Currently it is off by default because we've noticed performance regressions with it on, on-prem. This issue is for tracking and fixing the performance regressions.
The following issues are related, but may not fix the root-cause of the perf regression.
- #23526
- #23131
Contributor guide
Assessment
This issue has not been assessed yet.