apache / apache/gravitino

[EPIC] Add durable deletion-generation storage

Open
#12,220 2 comments 0 reactions 1 assignee Claimed by @nevzheng View on GitHub
epic
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Describe the proposal

This is the relational-storage child epic of #12219. It owns only the metadata representation and transactions required by soft deletion.

## Data model

- The existing `table_meta.table_id` remains the stable table identity.
- `table_meta.deleted_at` records when the row was deleted.
- Nullable `table_meta.deletion_id` is a logical pointer to the current active deletion action.
- `entity_deletion` stores only the opaque `deletion_id`, lifecycle `state`, fixed `retention_expires_at`, and optional future `purge_job_id`.
- Identity, parent, name, ownership, and deletion time remain authoritative on `table_meta`; they are not duplicated into the action.
- Table-owned child metadata remains attached through `table_id` and receives no duplicate deletion pointer.

## Transaction invariants

- DELETE inserts the action and stamps the exact live table root with `deleted_at` and `deletion_id` in one transaction. A losing root update rolls back the action insert.
- RESTORE locks the exact action and retained root, verifies that the action is recoverable, sets `deleted_at=0`, clears `deletion_id`, and removes the active action in one transaction.
- Coordination is limited to the affected rows; there are no table-wide database locks.
- Successful restore consumes the action. The storage model does not retain a terminal `RESTORED` receipt.
- Legacy metadata garbage collection skips table roots carrying an active `deletion_id`; future purge work owns their final removal.

## After this epic

The relational layer can represent, atomically retain, inspect, and reactivate an existing table row. No public API behavior or physical Iceberg cleanup is introduced here.

## Out of scope

- Iceberg REST DELETE, discovery, and UNDROP routes
- Same-name create/register enforcement
- Physical Iceberg cleanup
- Purge scheduling, workers, job progress, or operator APIs
- Deletion-specific audit storage

### Task list

- [ ] #12222 — schema, migrations, and persistent models
- [ ] #12223 — active deletion and restore transactions

Review order: #12222 → #12223. API integration continues with #12225 under #12221.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.