Copy-on-Write and Merge-on-Read support
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### What's the feature are you trying to implement?
`iceberg-rust` can append data files but has no support for row-level mutations. Query engines that need UPDATE, DELETE, MERGE, or compaction have no path through the transaction API today.
This epic tracks full Copy-on-Write (CoW) and Merge-on-Read (MoR) support. CoW comes first because it provides the foundation MoR depends on.
## Copy-on-Write
- [x] #2185 `OverwriteAction`--overwrite semantics with manifest rewriting for deleted entries
- [ ] Conflict detection (`validateNoConflictingData` / `validateNoConflictingDeletes`)
- [ ] Filter-based overwrite (`overwriteByRowFilter`)
- [ ] Compaction via rewrite
## Merge-on-Read
- [ ] `RowDeltaAction`--write position and equality delete files (write path)
- [ ] Scan-side delete file reconciliation--apply delete files at read time (read path)
- [ ] V3 Deletion Vectors (Puffin DVs)
## Related
- Java [`OverwriteFiles`](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/OverwriteFiles.java)
- Java [`RowDelta`](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/RowDelta.java)
- Existing `FastAppendAction` (#349)
### Willingness to contribute
I can contribute to this feature independently
Contributor guide
Research direction
Start by reading the existing OverwriteAction and FastAppendAction implementations, then compare their transaction APIs with the related Java OverwriteFiles and RowDelta APIs. The issue lists conflict detection, filter-based overwrite, compaction, RowDeltaAction, delete-file reconciliation, and deletion vectors as separate workstreams; completion depends on selecting and fully implementing one of them with corresponding coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100