apache / apache/gravitino

[EPIC] Idempotency-Key support for Iceberg REST server (multi-node safe)

Open
#12,048 0 comments 0 reactions 0 assignees View on GitHub
epic
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 17h
Merged PRs (30d)
339

Description

## Describe the proposal

The Iceberg REST spec defines an optional `Idempotency-Key` header (UUIDv7) on mutation endpoints. When present, the server must return the original response for replayed keys instead of re-executing the operation, preventing duplicate side-effects from client retries.

This epic covers the full implementation of idempotency support in Gravitino's Iceberg REST server, including multi-node consistency. The design accounts for:

- **Multi-node deployments**: Retries may hit different nodes behind a load balancer
- **Node failure resilience**: Cached responses must survive node crashes
- **Configurable storage backends**: In-memory (single-node/dev) and database-backed (production)
- **Spec compliance**: `idempotency-key-lifetime` advertised in `GET /v1/config`

Reference: [apache/polaris#3205](https://github.com/apache/polaris/pull/3205) (database-backed IdempotencyStore)

Supersedes #10683.

## Task list

- [ ] Design document: Idempotency-Key support (multi-node)
- [ ] IdempotencyStore SPI interface and in-memory implementation
- [ ] JDBC-backed IdempotencyStore using Gravitino's relational backend
- [ ] Header handling, replay logic, and endpoint integration
- [ ] Advertise idempotency-key-lifetime in GET /v1/config
- [ ] Unit and integration tests

Contributor guide

Open the contributing guide

Research direction

No files or tests are named. Start with the design document, then trace the Iceberg REST mutation endpoints and GET /v1/config alongside Gravitino's relational backend; done means the store SPI, in-memory and JDBC implementations, replay handling, configuration advertisement, and unit and integration tests are complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.