apache / apache/gravitino

[EPIC] Support metadata undelete for retained soft deletions

Open
#12,187 0 comments 0 reactions 1 assignee Claimed by @nevzheng View on GitHub
epic
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 17h
Merged PRs (30d)
339

Description

### Describe the proposal

Gravitino already retains soft-deleted relational metadata until garbage collection, but users cannot discover or recover it through the public API.

Add metadata-only undelete with consistent semantics across supported objects:

- Discover tombstones using `GET ?include=deleted`, optionally filtered by `name` or immutable `id`.
- Read an exact tombstone using `GET ?include=deleted&id=`, returning a strong ETag.
- Recover it using conditional `PATCH ?include=deleted&id=`, `If-Match`, and `{"deleted":false}`.
- Recover only the most recent eligible same-name deletion generation.
- Restore the exact recorded metadata deletion tree in one relational transaction.
- Make exact retries idempotent when successful replay is provable.
- Return typed failures for conflicts, expiry, stale ETags, and missing preconditions.
- Initially authorize deleted reads and recovery through `SERVICE_ADMIN`.

Supported roots include tables, functions, models, filesets, views, topics, schemas, catalogs, metalakes, policies, users, groups, roles, job templates, and tags.

This epic is metadata-only. Recovery does not call connectors, identity providers, authorization plugins, executors, staging systems, or other downstream systems. Reconciliation with those systems remains future work because preserving downstream data is safer than attempting an unsafe reconstruction.

Design proposal: **TBD**
POC pull request: **TBD**

Out of scope for this epic:

- Per-scope retention configuration
- Explicit hard-purge API
- Fine-grained recovery authorization
- External-system reconciliation
- Pagination
- Python, CLI, UI, and MCP surfaces

### Task list

- [ ] #12188 — Build metadata undelete foundations and the table reference flow
- [ ] #12189 — Add metadata undelete for catalog-scoped leaf objects
- [ ] #12190 — Support metadata undelete for container hierarchies
- [ ] #12191 — Support metadata undelete for metalake-scoped roots

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.