apache / apache/paimon

[Bug] Branch Base Snapshot Gets Expired After Deleting Referenced Tag

Open
#6,272 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

master

### Compute Engine

all

### Minimal reproduce step

## Summary
When a branch is created based on a tag, deleting that tag can cause the branch's base snapshot to be expired during snapshot expiration operations, making the branch unqueryable.

Currently, Paimon has a protection mechanism that prevents snapshots referenced by tags from being expired (implemented in `ExpireSnapshotsImpl.java`). However, there's no corresponding protection for branches that reference tags.

The issue occurs in the following scenario:
1. Create a tag pointing to a specific snapshot
2. Create a branch based on this tag
3. Delete the tag
4. Run snapshot expiration
5. The base snapshot of the branch gets expired because it's no longer protected by the tag
6. The branch becomes unqueryable

### What doesn't meet your expectations?

base snapshot not be deleted

### Anything else?

## Proposed Solutions

### Solution 1: Tag Deletion Protection (Preferred)

Add a protection mechanism when deleting tags to prevent deletion if any branches reference the tag.

**Implementation approach**:
- Modify `TagManager.deleteTag()` method
- Before deleting a tag, check if any branches reference this tag
- If branches exist, either:
- Reject the deletion with an error message
- Require a force flag to proceed with deletion

**Advantages**:
- Prevents the problem at the source
- Clear error message to users
- Maintains data integrity

### Solution 2: Branch-aware Snapshot Expiration

Modify the snapshot expiration logic to consider branches when determining which snapshots to protect.

**Implementation approach**:
- Modify `ExpireSnapshotsImpl.expireUntil()` method
- Add branch manager to collect all branch base snapshots
- Include branch-referenced snapshots in the skipping set

**Advantages**:
- More comprehensive protection
- Handles edge cases where branches might reference snapshots directly

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading ExpireSnapshotsImpl.java, especially expireUntil(), and TagManager.deleteTag(). Reproduce the tag, branch, tag-deletion, and snapshot-expiration sequence described in the issue. Done means the branch remains queryable after expiration, with the chosen protection behavior covered by the relevant existing tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.