[EPIC] Idempotency-Key support for Iceberg REST server (multi-node safe)
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 339
Description
## Describe the proposal
The Iceberg REST spec defines an optional `Idempotency-Key` header (UUIDv7) on mutation endpoints. When present, the server must return the original response for replayed keys instead of re-executing the operation, preventing duplicate side-effects from client retries.
This epic covers the full implementation of idempotency support in Gravitino's Iceberg REST server, including multi-node consistency and node-failure resilience.
Key concerns addressed:
- **Multi-node deployments**: Retries may hit different nodes behind a load balancer; a shared durable store (JDBC-backed) ensures all nodes see the same idempotency state
- **Node failure resilience**: DB-backed records survive crashes; TTL-based expiry handles abandoned reservations
- **Configurable storage backends**: In-memory (single-node/dev) and JDBC-backed (production)
- **Spec compliance**: UUIDv7 validation, `idempotency-key-lifetime` advertised in `GET /v1/config`
Reference: [apache/polaris#3205](https://github.com/apache/polaris/pull/3205) (database-backed IdempotencyStore)
Design document: `design-docs/iceberg-idempotency-key.md`
Supersedes #10683.
## Task list
- [ ] Design document review and approval
- [ ] IdempotencyStore SPI interface, IdempotencyRecord model, and InMemoryIdempotencyStore
- [ ] JdbcIdempotencyStore using Gravitino's relational backend (H2/MySQL/PostgreSQL)
- [ ] Header handling, replay logic, and mutation endpoint integration
- [ ] Advertise `idempotency-key-lifetime` in `GET /v1/config`
- [ ] Unit and integration tests
Contributor guide
Research direction
Start with design-docs/iceberg-idempotency-key.md and review the task list for the proposed IdempotencyStore SPI, record model, storage backends, endpoint integration, configuration, and tests. Done means the design is approved and all listed multi-node, failure-resilience, spec-compliance, and unit/integration-test work is complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mysql, postgresql
- Domain
- api, backend, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100