[C++][Parquet] Loading big parquet files leads to huge memory consumption
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
Loading big parquet files (23GB file) leads to high memory consumption, ending in OOM. Memory consumption surpass my machine limit of 64GB ram. Happens on Linux, arrow 17.0.0, Ubuntu 22
Reader is set up similar as demonstrated on docs page via `` parquet::arrow::FileReaderBuilder`` with key difference of
passing custom allocator. Huge usage of memory is not report by my custom allocator nor by memory pool passed to reader.
After invastigation iI encouter thath RangeReadeCache created lot of cache entries, 735 to be exact witch each about 33MiB of length. Each with it's future with use count 1.

Also for some reason this uses default memory pool

### Component(s)
C++
Contributor guide
Assessment
This issue has not been assessed yet.