apache / apache/iceberg-rust

EPIC: v3 Support Tracking

Open
#2,411 6 comments 5 reactions 0 assignees View on GitHub
enhancement epic
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

This tracks the progress on adding full v3 support to iceberg-rust.
Based on the pyiceberg tracking (https://github.com/apache/iceberg-python/issues/1818)
and iceberg-go tracking (https://github.com/apache/iceberg-go/issues/589).

New DataTypes

* Variant - PR #2188 open
* Non-shredded read/write
* Shredded read
* Shredded write
* Variant statistics (skip min/max on parent, allow on shredded children)
* Geometry / Geography
* https://github.com/apache/iceberg-rust/issues/1884

Default Values

* initial-default and write-default support

Row Lineage

* first-row-id tracking on manifest entries
* next-row-id monotonicity validation
* Row lineage assignment on write

Deletion Vectors

* DV read support (Puffin + Roaring bitmap)
* DV write support
* Scanner integration

Multi-arg Transforms

* Metadata read (partition spec with multi-arg transforms)
* Eval support (when spec stabilizes)

Misc v3 Spec Items

* Emit null current-snapshot-id for empty v3 metadata (not -1)
* Support zstd metadata compression codec
* Gate v3-only metadata fields when reading v1/v2
* https://github.com/apache/iceberg-rust/issues/1652

Related

* Iceberg spec v3 changes: https://iceberg.apache.org/spec/#version-3
* iceberg-go v3 tracking: https://github.com/apache/iceberg-go/issues/589
* pyiceberg v3 tracking: https://github.com/apache/iceberg-python/issues/1818
* Java variant shredding: https://github.com/apache/iceberg/pull/14297

### Describe the solution you'd like

These would be grouped and owned by one or more contributors.

Contributor guide

Open the contributing guide

Research direction

Start with the Iceberg spec v3 changes and compare the pyiceberg and iceberg-go tracking issues linked in this issue. Use the checklist sections—new data types, default values, row lineage, deletion vectors, transforms, and miscellaneous spec items—to identify one separately scoped contribution. Done means implementing and validating a specific unchecked v3 requirement; this epic does not name files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.