REST catalog: client support for atomic multi-table commits (POST /v1/{prefix}/transactions/commit)
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
The REST catalog spec defines `POST /v1/{prefix}/transactions/commit`, which applies changes to multiple tables in one atomic operation, and `CommitTableRequest` in this crate already notes that `identifier` "must be present for CommitTransactionRequest" — but there is currently no way to drive that endpoint from iceberg-rust.
Proposal:
- `Transaction::prepare_commit()` — run a transaction's actions and return the resulting `TableCommit` without sending it to a catalog (the per-table building block).
- `RestCatalog::commit_transaction(Vec)` — submit the group as one `CommitTransactionRequest`; the catalog applies all of the changes or none of them.
Kept on `RestCatalog` rather than the `Catalog` trait because multi-table atomicity is a catalog capability rather than a universal guarantee; whether the trait should expose a capability for it seems like a good follow-up discussion.
Use case: a CDC producer (Postgres → Iceberg) that commits row deltas to many tables plus a changelog table per cadence, and needs the group to land atomically so downstream readers never observe a half-applied cut. Validated against Lakekeeper, which implements the endpoint.
I have a working implementation with unit tests and will open a PR.
Contributor guide
Research direction
Start by locating Transaction, RestCatalog, CommitTableRequest, and the existing REST catalog commit tests. Trace how per-table transaction actions are currently prepared and how REST commits are submitted. Done means the proposed prepare_commit and commit_transaction flows are covered by unit tests and submit one atomic CommitTransactionRequest to POST /v1/{prefix}/transactions/commit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100