Support validate and related logic in Snapshot
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### Is your feature request related to a problem or challenge?
To detect conflicts, we need to have `validate` and related logic to scan manifests and validate against history.
References:
- java: https://github.com/apache/iceberg/blob/c2478968e65368c61799d8ca4b89506a61ca3e7c/core/src/main/java/org/apache/iceberg/BaseRowDelta.java#L128
- python: https://github.com/apache/iceberg-python/pull/1935
### Describe the solution you'd like
Add related logic to `SnapshotProduceOperation` so different actions like `RowDelta`, `OverwriteFiles`, and `RewriteFiles` can use and override (if needed)
### Willingness to contribute
I would be willing to contribute to this feature with guidance from the Iceberg Rust community
Contributor guide
Research direction
Start with the SnapshotProduceOperation entry point and compare the referenced Java BaseRowDelta logic with the linked Python implementation. Trace how RowDelta, OverwriteFiles, and RewriteFiles currently operate, then define the manifest and history validation behavior they should share or override. Done means these actions can use the related conflict-detection logic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100