[EPIC] Add Iceberg REST soft-delete lifecycle and recovery APIs
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### Describe the proposal
This is the Iceberg REST API child epic of #12219. It depends on the relational foundation in #12220.
The first API milestone is intentionally small:
```text
DELETE table → retain and hide the original row
GET tables?deleted=true → list authorized retained names
POST table/undrop → reactivate the selected original row
```
## API contract
### DELETE
Reuse the existing Iceberg REST table DELETE endpoint.
- When soft deletion is enabled, both `purgeRequested` values retain the original `table_meta` row and reserve its name.
- Return `204 No Content` only after the metadata transaction commits.
- A repeated DELETE of the retained generation returns `204` without allocating another action.
- Never unregister Iceberg metadata or delete files on the request thread.
- When soft deletion is disabled, preserve existing behavior.
### Deleted-table discovery
Reuse the existing table collection resource:
`GET /v1/{prefix}/namespaces/{namespace}/tables?deleted=true`
- Return the ordinary `ListTablesResponse`, containing authorized retained table names only.
- Reuse existing `pageToken` and `pageSize` behavior.
- Filter authorization before pagination and disclosure.
- Do not expose deletion IDs, lifecycle details, metadata locations, credentials, cleanup fields, or deletion-action ETags.
- Omitted `deleted` and `deleted=false` keep live-table behavior unchanged.
### UNDROP
Add an explicit, empty-body action:
`POST /management/v1/{prefix}/namespaces/{namespace}/tables/{table}/undrop`
- Resolve and authorize the retained row by name, then carry its exact internal `{table_id, deletion_id}` into the transaction.
- Do not require a public deletion ID, deletion-action ETag, or `If-Match`.
- Require the action to remain `DELETED`, unexpired, and unattached to a purge job.
- Reactivate the original `table_meta` row and consume the active action in one transaction.
- Return the ordinary live table response and live-table ETag.
- Use concealed `404` for missing or unauthorized targets, `409` for a conflicting state, and `410` after the UNDROP boundary.
## Shared invariants
- The retained row reserves its original name; create, register, and rename-to that name fail until restore or purge.
- DELETE and UNDROP use fine-grained row coordination, not table-wide locks.
- The internal deletion ID fences the exact generation but is not part of the public API.
- UNDROP directly reactivates the row. It does not create a replacement table, register Iceberg metadata, touch files, or run a saga.
## After this epic
Opt-in Iceberg REST soft deletion works through DELETE, deleted-name discovery, and name-only UNDROP. Physical cleanup and the durable purge-job system remain separate work.
## Out of scope
- Garbage-collector selection and physical file deletion
- Purge workers, progress APIs, and operator controls
- Real UNDROP-versus-worker race testing
- Large-scale cleanup and restart validation
### Task list
- [ ] #12225 — review 1/4: Iceberg REST soft-delete lifecycle
- [ ] #12228 — review 2/4: soft deletion through Iceberg REST DELETE
- [ ] #12226 — review 3/4: list soft-deleted Iceberg REST tables
- [ ] #12227 — review 4/4: UNDROP soft-deleted Iceberg REST tables
Review order: #12222 → #12223 → #12225 → #12228 → #12226 → #12227.
Contributor guide
Research direction
Start with the existing Iceberg REST table DELETE endpoint and table collection resource, then review the relational foundation in #12220 and the child issues in the stated review order. Confirm that DELETE, deleted-table discovery, and name-only UNDROP meet the listed authorization, transaction, lifecycle, and response requirements while leaving physical cleanup out of scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100