iceberg v3 has to set first-row-id
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### Is your feature request related to a problem or challenge?
Reading from the iceberg [spec](https://iceberg.apache.org/spec/#version-3), it mentions
- Writers must set the table's next-row-id and use the existing next-row-id as the first-row-id when creating new snapshots
+ When a table is upgraded to v3, next_row_id should be initialized to 0
+ When committing a new snapshot next-row-id must be incremented by at least the number of newly assigned row ids in the snapshot
+ It is recommended to increment next-row-id by the total added_rows_count and existing_rows_count of all manifests assigned a first_row_id
If I don't read it wrong, we should have `next_row_id` field in table metadata
But I don't see it in [`TableMetadata`](https://github.com/apache/iceberg-rust/blob/7362bbdcf72910d95ee88940f229d2eea40b3fa1/crates/iceberg/src/spec/table_metadata.rs#L137-L213), I'm wondering if we should add this field and set it on write?
### Describe the solution you'd like
_No response_
### Willingness to contribute
I can contribute to this feature independently
Contributor guide
Research direction
Start with crates/iceberg/src/spec/table_metadata.rs, then trace the v3 metadata and snapshot-writing paths that handle table commits. Compare those paths with the Iceberg v3 specification, and verify that next_row_id is represented, initialized on upgrade, and updated when snapshots assign row IDs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100