apache / apache/iceberg

The Metastore lock timeout led to the loss of metadata overwrite data.

Closed
#15,446 1 comment 0 reactions 0 assignees View on GitHub
bug stale
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 16h
Merged PRs (30d)
129

Description

### Apache Iceberg version

1.10.1 (latest release)

### Query engine

Spark

### Please describe the bug 🐞

1. We have a third-party service to clean up snapshot versions. During the cleaning process, if there is a long waiting time after the first confirmation of the lock's survival as shown;
2. During the waiting period, there were other normal submissions of batch 2026-02-08, and metadata_location was updated;
3. When the waiting for the third-party service ended, the metadata_location of batch 2026-02-08 was overwritten using the file of batch 2026-02-07 metadata_location.
4. This ultimately led to the data loss of batch 2026-02-08.

Image

### Willingness to contribute

- [x] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [x] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Research direction

Start with the reported Spark workflow and the sequence of lock-survival confirmation, waiting, and metadata_location updates. Reproduce the third-party snapshot-cleanup scenario against Iceberg 1.10.1, then trace the metastore lock and metadata overwrite behavior. Done means a delayed cleanup cannot overwrite newer metadata or lose the later batch's data.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.