Consider adding BloomFilter reading support to `ParquetMetadataReader`
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Parquet now has the wonderful [`ParquetMetaDataReader`](https://docs.rs/parquet/latest/parquet/file/metadata/struct.ParquetMetaDataReader.html) structure from @adriangb and @etseidl
This handles reading the footer metadata as well as the page indexes.
@progval noted in https://github.com/apache/arrow-rs/pull/6505#discussion_r1787879839 that BloomFilters are similiar to the PageIndex, but are not currently read/written by the ParquetMetaDataReader
**Describe the solution you'd like**
I would like to be able to configure the ParquetMetaDataReader (and writer) to read BloomFilters as well
**Describe alternatives you've considered**
This might look something like
```rust
// read parquet metadata including page indexes
let file = open_parquet_file("some_path.parquet");
let mut reader = ParquetMetaDataReader::new()
.with_bloom_filters(true);
reader.try_parse(&file).unwrap();
let metadata = reader.finish().unwrap();
// Somehow get access to the bloom filters (not sure what that API would look like)
```
**Additional context**
Contributor guide
Research direction
Start with the ParquetMetaDataReader entry point and the linked Arrow PR discussion about BloomFilters and PageIndex support. Trace how try_parse and finish expose metadata, then determine the reader and writer configuration and BloomFilter access needed for the feature to be complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100