[Subtask] Background reconcile to remove stale store rows for external-backed catalogs
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### Describe the subtask
A Gravitino store row can outlive its external object — after a swallowed/failed `store.delete` (`TableOperationDispatcher.dropTable` ignores `NoSuchEntityException`), a crash between the external op and the store write, or an out-of-band drop in Hive/JDBC. Today such stale rows are only repaired lazily on the next read, and actively removed only for schemas (`OrphanedSchemaCleanup`). There is no general active cleanup.
Pre-existing gap, independent of the TreeLock work.
### How to fix
Generalize the existing `OrphanedSchemaCleanup` / `SchemaEntityCleaner` pattern (external-existence probe -> delete orphaned store rows) from schemas to table/fileset/topic/model. Reuse `deleteTable`'s cascade so relations are removed with the row.
Guardrails (an external probe is slow and can flap):
- External-backed catalogs only — never managed catalogs, whose store is authoritative.
- Delete only after **N consecutive misses + a grace period**, to survive transient external outages.
- Rate-limit and run on a low-priority schedule (the `RelationalGarbageCollector` cadence fits).
- Tests: a stale row with the external object gone is reconciled; a transient external outage does **not** delete a live row.
### Parent issue
https://github.com/apache/gravitino/issues/10238
Contributor guide
Research direction
Start by reading OrphanedSchemaCleanup and SchemaEntityCleaner, then inspect TableOperationDispatcher.dropTable and the RelationalGarbageCollector cadence. Define reconciliation for external-backed table, fileset, topic, and model rows using consecutive misses, a grace period, and rate limiting. Done means a missing external object is cleaned with its relations, while a transient outage leaves a live row intact; add tests for both cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100