apache / apache/gravitino

[Improvement] Prevent stale Lance schema repair from overwriting newer metadata

Open
#12,957 0 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### What would you like to be improved?

Schema repair can capture dataset version 8, then overwrite metadata already updated to version 9 with the older schema/version. Metadata-store CAS does not validate the earlier dataset observation.

Parent: #9087. Follow-up from the review of #12954.

### How should we improve?

Validate the observed table identity, location, and version before applying repair. Re-read when they change or a concurrency conflict occurs instead of replaying stale state. A version-only comparison is insufficient across replacement or location changes.

Add a deterministic regression test for the version-8/version-9 interleaving, replacement/location changes, and concurrent repair. Include integration coverage that verifies reads cannot persist stale schema metadata.

Contributor guide

Open the contributing guide

Research direction

Start by tracing schema repair, metadata-store CAS, and the read path described in the issue. Add deterministic coverage for the version-8/version-9 interleaving, replacement or location changes, and concurrent repair, plus integration coverage proving reads cannot persist stale schema metadata.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.