NVIDIA / NVIDIA/cudf

[FEA] Expose TotalBytesRead in Parquet Reader

Open
#19,085 0 comments 0 reactions 0 assignees View on GitHub
feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

I’d like to propose and discuss adding a metric to the Parquet reader to track the total bytes read from the file during I/O operations. This metric would capture the raw, encoded/compressed bytes transferred from storage/io-source.

The motivation behind this is twofold: first, to provide visibility into the actual data movement measurement at the storage layer, which is helpful for optimizing I/O-bound workflows or profiling. Second, it would enable users to profile and compare the effect of Parquet pruning.

**Describe the solution you'd like**

Currently, `table_metadata` tracks rowgroups but not byte counts.

https://github.com/rapidsai/cudf/blob/a664c1464fef02185350e65a80868ecec59dacb1/cpp/include/cudf/io/types.hpp#L290-L297

Adding a `read_bytes` field to `cpp/include/cudf/io/types.hpp` would align with this existing pattern. The value could come directly from KvikIO, which already tracks raw bytes read. This would require minimal overhead in this book-keeping.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.