apache / apache/gravitino

[Subtask] Decouple OCC versions from fileset and policy history versions

Open
#12,206 0 comments 0 reactions 0 assignees View on GitHub
subtask
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 15h
Merged PRs (30d)
315

Description

### Describe the subtask

Fileset and policy currently reuse history versions for concurrency control. Since history versions do not advance for every metadata mutation, their OCC implementations still require full-row comparisons. Introduce dedicated OCC versions, or otherwise separate concurrency versions from history revisions, including schema migration and concurrency tests.

### Parent issue

https://github.com/apache/gravitino/issues/12166

### Additional scope: model_meta insert does not write the OCC version columns

`ModelMetaBaseSQLProvider.insertModelMeta` and `insertModelMetaOnDuplicateKeyUpdate` do not list `current_version` / `last_version` in their column lists, so inserted rows rely on the schema `DEFAULT 1`. `POConverters.initializeModelPO` does set both to `INIT_VERSION`, and that value is silently discarded. Model is a special case — its `current_version` is a pure OCC token with no version-history table behind it — but the insert should still write the columns explicitly, like every other entity does, so the PO stays the single source of truth.

Related: the same coupling problem in reverse affects table/view/function, whose `current_version` doubles as the join key into `*_version_info`. Their upserts therefore cannot simply advance the version (see the discussion in #12168), which is the core of this subtask.

Contributor guide

Open the contributing guide

Research direction

Start with the parent issue #12166 and the discussion referenced in #12168 to understand the intended version model. Then inspect ModelMetaBaseSQLProvider.insertModelMeta, insertModelMetaOnDuplicateKeyUpdate, and POConverters.initializeModelPO, along with the schema migration and concurrency tests. Done means dedicated OCC versions are separated from history revisions and the model inserts explicitly persist both version columns.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.