Allow `write.metadata.previous-versions-max` table property to be 0 for empty metadata logs
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Feature Request / Improvement
`write.metadata.previous-versions-max` is currently floored to 1:
https://github.com/apache/iceberg/blob/70b90330582e4023a097330620a3e63746b2375b/core/src/main/java/org/apache/iceberg/TableMetadata.java#L1801-L1807
so a table always retains at least one previous metadata file in the `metadata-log`.
I was a bit confused by this, because I don't see a concrete reason to disallow _empty_ metadata logs; I'd therefore like to know what folks think about permitting a value of 0 instead to enable this.
With [`write.metadata.delete-after-commit.enabled=true`](https://iceberg.apache.org/docs/latest/maintenance/#remove-old-metadata-files), this would then delete the superseded metadata file on every commit.
(My use case for this is from a REST catalog principle-based standpoint where an empty metadata log would allow metadata JSON to be "self-contained" and only reference its current snapshots, instead of indirectly reference snapshots in prior metadata files too)
### Query engine
None
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
Contributor guide
Research direction
Start in core/src/main/java/org/apache/iceberg/TableMetadata.java at the linked lines that floor write.metadata.previous-versions-max to 1. Trace the metadata-log retention behavior and its tests, including the interaction with write.metadata.delete-after-commit.enabled. Done means a value of 0 permits an empty metadata log and the described deletion behavior is covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100