apache / apache/iceberg-rust

Copy-on-Write and Merge-on-Read support

Open
#2,186 5 comments 1 reaction 0 assignees View on GitHub
epic
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### What's the feature are you trying to implement?

`iceberg-rust` can append data files but has no support for row-level mutations. Query engines that need UPDATE, DELETE, MERGE, or compaction have no path through the transaction API today.

This epic tracks full Copy-on-Write (CoW) and Merge-on-Read (MoR) support. CoW comes first because it provides the foundation MoR depends on.

## Copy-on-Write

- [x] #2185 `OverwriteAction`--overwrite semantics with manifest rewriting for deleted entries
- [ ] Conflict detection (`validateNoConflictingData` / `validateNoConflictingDeletes`)
- [ ] Filter-based overwrite (`overwriteByRowFilter`)
- [ ] Compaction via rewrite

## Merge-on-Read

- [ ] `RowDeltaAction`--write position and equality delete files (write path)
- [ ] Scan-side delete file reconciliation--apply delete files at read time (read path)
- [ ] V3 Deletion Vectors (Puffin DVs)

## Related

- Java [`OverwriteFiles`](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/OverwriteFiles.java)
- Java [`RowDelta`](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/RowDelta.java)
- Existing `FastAppendAction` (#349)

### Willingness to contribute

I can contribute to this feature independently

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing OverwriteAction and FastAppendAction implementations, then compare their transaction APIs with the related Java OverwriteFiles and RowDelta APIs. The issue lists conflict detection, filter-based overwrite, compaction, RowDeltaAction, delete-file reconciliation, and deletion vectors as separate workstreams; completion depends on selecting and fully implementing one of them with corresponding coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.