Add `RewriteFiles` support to iceberg-rust
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### Is your feature request related to a problem or challenge?
Currently, `iceberg-rust` does not support committing operations that both add and delete files. This feature, known as `RewriteFiles` in `iceberg-java`, is crucial for our use case, particularly for compaction.
### Describe the solution you'd like
Based on my initial investigation in #1606, I believe the following three components are necessary to implement `RewriteFiles` in `iceberg-rust`:
- **Process delete files**: We need to add logic to handle delete files when generating snapshots.
- **Implement `RewriteFilesAction`**: This will require creating a new transaction action, `RewriteFilesAction`, and related classes like `RewriteFilesOperation`.
- **Support validation**: We need to add validation for rewrite operations to prevent conflicts. Currently we don't have any validation mechnism in the codebase, so a new trait `SnapshotValidator` would be needed.
### Willingness to contribute
I would be willing to contribute to this feature with guidance from the Iceberg Rust community
Contributor guide
Assessment
This issue has not been assessed yet.