NVIDIA / NVIDIA/cudf

[FEA] Improve I/O performance for bloom filters and dictionary pages in PQ reader

Open
#21,398 0 comments 0 reactions 0 assignees View on GitHub
feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
We have recently made some improvements to column chunks I/O performance in the parquet reader. See #21360, #21323 which need to be propagated to I/O for bloom filters and dictionary pages during row group filtering. Even better if we can separate out a generic common I/O function that serves all three needs (or at least move all of them to `parquet_io_utils.cpp` for easier future updates.

**Describe the solution you'd like**
* I/O improvements for column chunks propagated to bloom filters and dictionary pages.
* Move all I/O functions to a common place such as parquet_io_utils.hpp/cpp and use common functions if possible.

**Describe alternatives you've considered**
Manually propagate changes to all said places

**Additional context**
N/A

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.