apache / apache/iceberg-rust

iceberg v3 has to set first-row-id

Open
#1,652 4 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### Is your feature request related to a problem or challenge?

Reading from the iceberg [spec](https://iceberg.apache.org/spec/#version-3), it mentions
- Writers must set the table's next-row-id and use the existing next-row-id as the first-row-id when creating new snapshots
+ When a table is upgraded to v3, next_row_id should be initialized to 0
+ When committing a new snapshot next-row-id must be incremented by at least the number of newly assigned row ids in the snapshot
+ It is recommended to increment next-row-id by the total added_rows_count and existing_rows_count of all manifests assigned a first_row_id

If I don't read it wrong, we should have `next_row_id` field in table metadata
But I don't see it in [`TableMetadata`](https://github.com/apache/iceberg-rust/blob/7362bbdcf72910d95ee88940f229d2eea40b3fa1/crates/iceberg/src/spec/table_metadata.rs#L137-L213), I'm wondering if we should add this field and set it on write?

### Describe the solution you'd like

_No response_

### Willingness to contribute

I can contribute to this feature independently

Contributor guide

Open the contributing guide

Research direction

Start with crates/iceberg/src/spec/table_metadata.rs, then trace the v3 metadata and snapshot-writing paths that handle table commits. Compare those paths with the Iceberg v3 specification, and verify that next_row_id is represented, initialized on upgrade, and updated when snapshots assign row IDs.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.