apache / apache/gravitino

[EPIC] Add Iceberg REST soft-delete lifecycle and recovery APIs

Open
#12,221 0 comments 0 reactions 0 assignees View on GitHub
epic
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Describe the proposal

This is the Iceberg REST API child epic of #12219. It depends on the relational foundation in #12220.

The first API milestone is intentionally small:

```text
DELETE table → retain and hide the original row
GET tables?deleted=true → list authorized retained names
POST table/undrop → reactivate the selected original row
```

## API contract

### DELETE

Reuse the existing Iceberg REST table DELETE endpoint.

- When soft deletion is enabled, both `purgeRequested` values retain the original `table_meta` row and reserve its name.
- Return `204 No Content` only after the metadata transaction commits.
- A repeated DELETE of the retained generation returns `204` without allocating another action.
- Never unregister Iceberg metadata or delete files on the request thread.
- When soft deletion is disabled, preserve existing behavior.

### Deleted-table discovery

Reuse the existing table collection resource:

`GET /v1/{prefix}/namespaces/{namespace}/tables?deleted=true`

- Return the ordinary `ListTablesResponse`, containing authorized retained table names only.
- Reuse existing `pageToken` and `pageSize` behavior.
- Filter authorization before pagination and disclosure.
- Do not expose deletion IDs, lifecycle details, metadata locations, credentials, cleanup fields, or deletion-action ETags.
- Omitted `deleted` and `deleted=false` keep live-table behavior unchanged.

### UNDROP

Add an explicit, empty-body action:

`POST /management/v1/{prefix}/namespaces/{namespace}/tables/{table}/undrop`

- Resolve and authorize the retained row by name, then carry its exact internal `{table_id, deletion_id}` into the transaction.
- Do not require a public deletion ID, deletion-action ETag, or `If-Match`.
- Require the action to remain `DELETED`, unexpired, and unattached to a purge job.
- Reactivate the original `table_meta` row and consume the active action in one transaction.
- Return the ordinary live table response and live-table ETag.
- Use concealed `404` for missing or unauthorized targets, `409` for a conflicting state, and `410` after the UNDROP boundary.

## Shared invariants

- The retained row reserves its original name; create, register, and rename-to that name fail until restore or purge.
- DELETE and UNDROP use fine-grained row coordination, not table-wide locks.
- The internal deletion ID fences the exact generation but is not part of the public API.
- UNDROP directly reactivates the row. It does not create a replacement table, register Iceberg metadata, touch files, or run a saga.

## After this epic

Opt-in Iceberg REST soft deletion works through DELETE, deleted-name discovery, and name-only UNDROP. Physical cleanup and the durable purge-job system remain separate work.

## Out of scope

- Garbage-collector selection and physical file deletion
- Purge workers, progress APIs, and operator controls
- Real UNDROP-versus-worker race testing
- Large-scale cleanup and restart validation

### Task list

- [ ] #12225 — review 1/4: Iceberg REST soft-delete lifecycle
- [ ] #12228 — review 2/4: soft deletion through Iceberg REST DELETE
- [ ] #12226 — review 3/4: list soft-deleted Iceberg REST tables
- [ ] #12227 — review 4/4: UNDROP soft-deleted Iceberg REST tables

Review order: #12222 → #12223 → #12225 → #12228 → #12226 → #12227.

Contributor guide

Open the contributing guide

Research direction

Start with the existing Iceberg REST table DELETE endpoint and table collection resource, then review the relational foundation in #12220 and the child issues in the stated review order. Confirm that DELETE, deleted-table discovery, and name-only UNDROP meet the listed authorization, transaction, lifecycle, and response requirements while leaving physical cleanup out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.