EPIC: v3 Support Tracking
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
This tracks the progress on adding full v3 support to iceberg-rust.
Based on the pyiceberg tracking (https://github.com/apache/iceberg-python/issues/1818)
and iceberg-go tracking (https://github.com/apache/iceberg-go/issues/589).
New DataTypes
* Variant - PR #2188 open
* Non-shredded read/write
* Shredded read
* Shredded write
* Variant statistics (skip min/max on parent, allow on shredded children)
* Geometry / Geography
* https://github.com/apache/iceberg-rust/issues/1884
Default Values
* initial-default and write-default support
Row Lineage
* first-row-id tracking on manifest entries
* next-row-id monotonicity validation
* Row lineage assignment on write
Deletion Vectors
* DV read support (Puffin + Roaring bitmap)
* DV write support
* Scanner integration
Multi-arg Transforms
* Metadata read (partition spec with multi-arg transforms)
* Eval support (when spec stabilizes)
Misc v3 Spec Items
* Emit null current-snapshot-id for empty v3 metadata (not -1)
* Support zstd metadata compression codec
* Gate v3-only metadata fields when reading v1/v2
* https://github.com/apache/iceberg-rust/issues/1652
Related
* Iceberg spec v3 changes: https://iceberg.apache.org/spec/#version-3
* iceberg-go v3 tracking: https://github.com/apache/iceberg-go/issues/589
* pyiceberg v3 tracking: https://github.com/apache/iceberg-python/issues/1818
* Java variant shredding: https://github.com/apache/iceberg/pull/14297
### Describe the solution you'd like
These would be grouped and owned by one or more contributors.
Contributor guide
Research direction
Start with the Iceberg spec v3 changes and compare the pyiceberg and iceberg-go tracking issues linked in this issue. Use the checklist sections—new data types, default values, row lineage, deletion vectors, transforms, and miscellaneous spec items—to identify one separately scoped contribution. Done means implementing and validating a specific unchecked v3 requirement; this epic does not name files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100