[EPIC] Implement Missing Write Actions
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
iceberg-rust currently supports FastAppendAction for simple appends but is missing several transaction actions available in the [Java Iceberg API](https://github.com/apache/iceberg/tree/main/api/src/main/java/org/apache/iceberg) that are needed for complete write support.
Java API (api/src/main/java/org/apache/iceberg/) | iceberg-rust (crates/iceberg/src/transaction/) | Status
-- | -- | --
AppendFiles.java | append.rs (FastAppendAction) | Available
OverwriteFiles.java | - | Missing
ReplacePartitions.java | - | Missing
DeleteFiles.java | - | Missing
RowDelta.java | - | Missing
RewriteFiles.java | - | Missing
### TODOs:
- [ ] Implement OverwriteAction - needed for INSERT OVERWRITE (static partition overwrite)
- [ ] Implement ReplacePartitionsAction - needed for dynamic partition overwrite
- [ ] Implement DeleteFilesAction - needed for DELETE FROM table operations
- [ ] Implement RowDeltaAction - needed for row-level DELETE, UPDATE, MERGE INTO (CoW/MoR)
- [ ] Implement RewriteFilesAction - needed for compaction and file rewriting
### Related open issues:
#1104 - Support RowDeltaAction
#2202 - Implement RowDeltaAction for row-level modifications (CoW)
#2201 - Add MERGE INTO support for DataFusion integration
#2205 - SQL UPDATE support for DataFusion integration
#2186 - Copy-on-Write and Merge-on-Read support
#1382 - [EPIC] Support for appending data to iceberg table
#2218 - Support for a DeltaWriter
### Motivation:
These transaction actions are required for [apache/datafusion-comet](https://github.com/apache/datafusion-comet) to implement a fully native Iceberg write path ([datafusion-comet#3595](https://github.com/apache/datafusion-comet/issues/3595), [datafusion-comet#2967](https://github.com/apache/datafusion-comet/issues/2967)). Comet currently writes Parquet data files natively via Rust/DataFusion but still relies on the Java Iceberg layer for table commits. Completing these actions in iceberg-rust would enable end-to-end native Iceberg writes - from Arrow batches through DataFusion execution to Iceberg table commits - eliminating JVM dependency for the entire write path.
Contributor guide
Research direction
Start by reading crates/iceberg/src/transaction/append.rs and comparing FastAppendAction with the corresponding classes under api/src/main/java/org/apache/iceberg/. Use the listed related issues to identify existing work, then address the missing actions; done means Overwrite, ReplacePartitions, DeleteFiles, RowDelta, and RewriteFiles support the native Iceberg write path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100