apache / apache/paimon

[Bug] set `sink.savepoint.auto-tag=true`, when I trigger a savepoint, cannot create a tag named `savepoint-${savepointID}`

Open
#4,677 5 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

paimon-flink-1.18-1.0-SNAPSHOT.jar

### Compute Engine

flink

### Minimal reproduce step

``` shell
flink run -m yarn-cluster --yarnname test_blob_tag_hourly \
/paimon/paimon-flink-action-1.0-20241127.002725-59.jar \
mysql_sync_table \
--warehouse s3://user/hive/warehouse \
--database paimon_db \
--table test_tag_hourly \
--primary_keys id \
--mysql_conf hostname=localhost \
--mysql_conf username=user \
--mysql_conf password=password \
--mysql_conf database-name='test' \
--mysql_conf table-name='test_tag' \
--mysql-conf server-time-zone='Atlantic/Reykjavik' \
--catalog-conf metastore=hive \
--catalog-conf hive-conf-dir=/etc/hive/conf.dist \
--catalog-conf warehouse=s3://user/hive/warehouse \
--catalog-conf lock.enabled=false \
--table-conf tag.automatic-creation=process-time \
--table-conf tag.creation-period=hourly \
--table-conf tag.creation-delay=3m \
--table-conf tag.num-retained-max=3 \
--table-conf sink.savepoint.auto-tag=true \
--table-conf metastore.tag-to-partition=hh \
--table-conf changelog-producer=input \
--table-conf sink.parallelism=2
```
and then trigger a savepoint

``` shell
flink savepoint :jobid s3://user/savepoint -yid application_id
```

select tags

``` sql
SELECT * FROM test_tag_hourly$tags;
```
can't find a tag named `savepoint-${savepointID}`

### What doesn't meet your expectations?

can't find a tag named `savepoint-${savepointID}`

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the issue with the provided Flink job and savepoint command, then inspect the savepoint auto-tag handling and related tag metadata flow. Verify the result by querying the table's $tags relation and confirming that a tag named savepoint-${savepointID} is present.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.