apache / apache/iceberg-rust

Parquet writer ignores per-column `write.parquet.bloom-filter-*` table properties

Open
#3,147 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### Is your feature request related to a problem or challenge?

`ParquetWriterBuilder::from_table_properties` translates `write.parquet.*` settings into `parquet-rs` WriterProperties (crates/iceberg/src/writer/file_writer/parquet_writer.rs:90-113): CDC, compression codec/level, row-group bytes, page size, page row limit, dict page size.

The bloom-filter properties are not among them, and crates/iceberg/src/spec/table_properties.rs defines no constants for them. A table that requests bloom filters gets files written without any. Reference behaviour is in TableProperties.java:184-199, consumed at parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java:652-668.

### Describe the solution you'd like

the three per-column properties (`write.parquet.bloom-filter-enabled.column.`, `write.parquet.bloom-filter-fpp.column.` and `write.parquet.bloom-filter-ndv.column.`) map directly onto parquet-rs.

### Willingness to contribute

I would be willing to contribute to this feature with guidance from the Iceberg Rust community

Contributor guide

Open the contributing guide

Research direction

Start in crates/iceberg/src/writer/file_writer/parquet_writer.rs:90-113 and crates/iceberg/src/spec/table_properties.rs, then compare the reference behavior in TableProperties.java:184-199 and parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java:652-668. Done means the three per-column bloom-filter properties are defined and mapped to parquet-rs WriterProperties.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.