[Subtask] Implement version-CAS OCC for entity update and soft-delete
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### Describe the subtask
Implement real optimistic concurrency control (version CAS) on the Gravitino store's entity write paths, per the OCC requirements in the design doc (PR #12157). Today no entity does a clean version CAS: `alter` (UPDATE) carries `current_version` in its `WHERE` but **freezes** the version for metalake/catalog/schema/topic (only `table` increments) and leans on a fragile full-row compare; `drop` (soft-delete) has **no** version predicate at all (`WHERE id=? AND deleted_at=0`, and the mapper takes only the id).
### Scope
1. **alter — always raise the version.** Every OCC-covered entity's successful UPDATE must set `current_version = old + 1`, and the UPDATE predicate is reduced to `id = ? AND current_version = ? AND deleted_at = 0` (drop the `audit_info`/name/... full-row compare). Frozen today: metalake, catalog, schema, topic; conditional: fileset, policy; already increments: table (still has a full-row `WHERE` to slim).
2. **drop — CAS delete.** Add an expected-version parameter to the soft-delete path; change the SQL to `UPDATE ... SET deleted_at= WHERE id=? AND current_version=? AND deleted_at=0`. (Today `softDelete...ById(Long id)` takes only the id.)
3. **drop — report 0 vs 1 rows.** The soft-delete returns the affected-row count so the service can tell **1 = deleted** from **0 = version changed / already gone**. Map 0 rows by re-reading: a version mismatch -> OCC conflict (retry -> HTTP 409); an already-deleted row -> `NoSuchEntity` / idempotent `false`, per the API contract.
4. **Backward compatibility with existing data.**
- **No schema migration**: `current_version`/`last_version` already exist (DEFAULT 1). Existing rows keep their value; the first OCC write reads-then-CAS's on it (frozen rows sitting at `current_version=1` bump to 2 on first write — no backfill needed).
- **Rolling upgrade / mixed nodes**: an old-code node's write does **not** bump the version, so a new-code node's CAS would not detect it. Gate the change (feature flag, or a single-release cutover with a documented note) so a cluster is never half-CAS / half-frozen.
- **Ordering**: land "always increment version" first, then slim the `WHERE`; never remove the full-row compare while any path still depends on it.
- **Out of scope of CAS**: `put(overwrite=true)` on create/import/reconcile stays a blind upsert — CAS applies only to update and soft-delete.
### Acceptance criteria
- alter/alter lost-update: two concurrent alters — one wins (`N->N+1`), the other matches 0 rows -> OCC conflict.
- drop CAS: stale-version drop returns 0 rows -> reported as conflict; current-version drop returns 1 row -> success.
- alter/drop: current-version drop wins; the concurrent alter then matches 0 rows (via `deleted_at`) — no lost write, no stale row.
- change-then-change-back (A->B->A) is caught by version, not by JSON byte equality.
- existing rows (`current_version=1`) upgrade cleanly with no migration.
- tests pass on MySQL and PostgreSQL.
### Relevant code
- `POConverters.update*POWithVersion` — version freeze/increment
- `*MetaBaseSQLProvider.update*Meta` (alter `WHERE`) and `softDelete*ById` (drop `WHERE`)
- `*MetaService.delete*` — pass the version, interpret the row count
### Parent issue
https://github.com/apache/gravitino/issues/10238
Contributor guide
Assessment
This issue has not been assessed yet.