apache / apache/iceberg-rust

[EPIC] Implement Missing Write Actions

Open
#2,269 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

iceberg-rust currently supports FastAppendAction for simple appends but is missing several transaction actions available in the [Java Iceberg API](https://github.com/apache/iceberg/tree/main/api/src/main/java/org/apache/iceberg) that are needed for complete write support.

Java API (api/src/main/java/org/apache/iceberg/) | iceberg-rust (crates/iceberg/src/transaction/) | Status
-- | -- | --
AppendFiles.java | append.rs (FastAppendAction) | Available
OverwriteFiles.java | - | Missing
ReplacePartitions.java | - | Missing
DeleteFiles.java | - | Missing
RowDelta.java | - | Missing
RewriteFiles.java | - | Missing

### TODOs:

- [ ] Implement OverwriteAction - needed for INSERT OVERWRITE (static partition overwrite)
- [ ] Implement ReplacePartitionsAction - needed for dynamic partition overwrite
- [ ] Implement DeleteFilesAction - needed for DELETE FROM table operations
- [ ] Implement RowDeltaAction - needed for row-level DELETE, UPDATE, MERGE INTO (CoW/MoR)
- [ ] Implement RewriteFilesAction - needed for compaction and file rewriting

### Related open issues:

#1104 - Support RowDeltaAction
#2202 - Implement RowDeltaAction for row-level modifications (CoW)
#2201 - Add MERGE INTO support for DataFusion integration
#2205 - SQL UPDATE support for DataFusion integration
#2186 - Copy-on-Write and Merge-on-Read support
#1382 - [EPIC] Support for appending data to iceberg table
#2218 - Support for a DeltaWriter

### Motivation:

These transaction actions are required for [apache/datafusion-comet](https://github.com/apache/datafusion-comet) to implement a fully native Iceberg write path ([datafusion-comet#3595](https://github.com/apache/datafusion-comet/issues/3595), [datafusion-comet#2967](https://github.com/apache/datafusion-comet/issues/2967)). Comet currently writes Parquet data files natively via Rust/DataFusion but still relies on the Java Iceberg layer for table commits. Completing these actions in iceberg-rust would enable end-to-end native Iceberg writes - from Arrow batches through DataFusion execution to Iceberg table commits - eliminating JVM dependency for the entire write path.

Contributor guide

Open the contributing guide

Research direction

Start by reading crates/iceberg/src/transaction/append.rs and comparing FastAppendAction with the corresponding classes under api/src/main/java/org/apache/iceberg/. Use the listed related issues to identify existing work, then address the missing actions; done means Overwrite, ReplacePartitions, DeleteFiles, RowDelta, and RewriteFiles support the native Iceberg write path.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, rust
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.