apache / apache/iceberg-rust

Support validate and related logic in Snapshot

Open
#1,344 3 comments 0 reactions 0 assignees View on GitHub
enhancement not-stale
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### Is your feature request related to a problem or challenge?

To detect conflicts, we need to have `validate` and related logic to scan manifests and validate against history.

References:
- java: https://github.com/apache/iceberg/blob/c2478968e65368c61799d8ca4b89506a61ca3e7c/core/src/main/java/org/apache/iceberg/BaseRowDelta.java#L128
- python: https://github.com/apache/iceberg-python/pull/1935

### Describe the solution you'd like

Add related logic to `SnapshotProduceOperation` so different actions like `RowDelta`, `OverwriteFiles`, and `RewriteFiles` can use and override (if needed)

### Willingness to contribute

I would be willing to contribute to this feature with guidance from the Iceberg Rust community

Contributor guide

Open the contributing guide

Research direction

Start with the SnapshotProduceOperation entry point and compare the referenced Java BaseRowDelta logic with the linked Python implementation. Trace how RowDelta, OverwriteFiles, and RewriteFiles currently operate, then define the manifest and history validation behavior they should share or override. Done means these actions can use the related conflict-detection logic.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.