[Bug] iceberg-rust does not respect compression settings for metadata & avro
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### Apache Iceberg Rust version
0.6
### Describe the bug
I'm using the Nessie catalog with the following settings:
```
- nessie.catalog.iceberg-config-defaults.write.avro.compression-level=3
- nessie.catalog.iceberg-config-defaults.write.avro.compression-codec=zstd
- nessie.catalog.iceberg-config-defaults.write.metadata.compression-codec=gzip
```
These settings are passed to the clients, but `iceberg-rust` does not have code that respects them.
As a result, it keeps producing uncompressed metadata and Avro files.
### To Reproduce
_No response_
### Expected behavior
_No response_
### Willingness to contribute
None
Contributor guide
Research direction
Start by tracing how the Nessie configuration keys write.avro.compression-level, write.avro.compression-codec, and write.metadata.compression-codec reach the metadata and Avro writers. Confirm the affected output currently lacks compression, then verify that the configured codecs and level are respected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100