apache / apache/paimon

[Bug]Flink deletes data error in paimon

Open
#6,137 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

paimon 0.8

### Compute Engine

flink 1.17.2

### Minimal reproduce step

paimon table :

```
CREATE TABLE paimon.hainadw.yydw_dm_entity_person_paimon_mini(
`hdid` string,
`attribute` string,
`last_update_dt` string,
`proctime` AS PROCTIME(),
PRIMARY KEY(`hdid`) NOT ENFORCED
) COMMENT ''
WITH (
'num-sorted-run.stop-trigger' = '2147483647',
'delete.force-produce-changelog' = 'false',
'changelog-producer' = 'input',
'sort-spill-threshold' = '10',
'bucket' = '64',
'log.scan.remove-normalize' = 'true',
'snapshot.time-retained' = '24h',
'log.consistency' = 'eventual',
'metastore.partitioned-table' = 'true'
)

```
There are two flink tasks writing to this table:

1 flink stream sql:
```
insert into paimon.hainadw.yydw_dm_entity_person_paimon_mini
select
hdid,
CleanJsonString(attribute),
date_format(current_date, 'yyyy-MM-dd') as last_update_dt
from new_device_after_fetch
where status = 'SUCCESS'
and attribute > '';

```

2 flink batch sql:
```
delete from paimon.hainadw.yydw_dm_entity_person_paimon_mini/*+ OPTIONS('write-only'='true') */
where last_update_dt < TIMESTAMPADD(DAY, -366, CURRENT_DATE);
```

### What doesn't meet your expectations?

2025-08-23 08:07:07,925 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_0) switched from RUNNING to FAILED on container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473).
java.lang.RuntimeException: File deletion conflicts detected! Give up committing.

Don't panic!
Conflicts during commits are normal and this failure is intended to resolve the conflicts.
Conflicts are mainly caused by the following scenarios:
1. Your job is suffering from back-pressuring.
There are too many snapshots waiting to be committed and an exception occurred during the commit procedure (most probably due to checkpoint timeout).
See https://paimon.apache.org/docs/master/maintenance/write-performance/ for how to improve writing performance.
2. Multiple jobs are writing into the same partition at the same time, or you use STATEMENT SET to execute multiple INSERT statements into the same Paimon table.
You'll probably see different base commit user and current commit user below.
You can use https://paimon.apache.org/docs/master/maintenance/dedicated-compaction#dedicated-compaction-jobto support multiple writing.
3. You're recovering from an old savepoint, or you're creating multiple jobs from a savepoint.
The job will fail continuously in this scenario to protect metadata from corruption.
You can either recover from the latest savepoint, or you can revert the table to the snapshot corresponding to the old savepoint.

Base commit user is: 5a90b504-8a73-43dc-871a-2e82d388ab37; Current commit user is: 5a90b504-8a73-43dc-871a-2e82d388ab37

Base entries are:
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-0.orc', minKey=org.apache.paimon.data.BinaryRow@8689bba1, maxKey=org.apache.paimon.data.BinaryRow@c328c0a1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-1.orc', minKey=org.apache.paimon.data.BinaryRow@c020a121, maxKey=org.apache.paimon.data.BinaryRow@bcc86d21}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-2.orc', minKey=org.apache.paimon.data.BinaryRow@60bffc9f, maxKey=org.apache.paimon.data.BinaryRow@87e819e1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-3.orc', minKey=org.apache.paimon.data.BinaryRow@f8b5b2a1, maxKey=org.apache.paimon.data.BinaryRow@831b2521}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-4.orc', minKey=org.apache.paimon.data.BinaryRow@c1a152e1, maxKey=org.apache.paimon.data.BinaryRow@b06c47e1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-5.orc', minKey=org.apache.paimon.data.BinaryRow@f40617e1, maxKey=org.apache.paimon.data.BinaryRow@69797d5f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-6.orc', minKey=org.apache.paimon.data.BinaryRow@1196e39f, maxKey=org.apache.paimon.data.BinaryRow@dde62d61}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-7.orc', minKey=org.apache.paimon.data.BinaryRow@ac5e4ca1, maxKey=org.apache.paimon.data.BinaryRow@fa627ea1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-8.orc', minKey=org.apache.paimon.data.BinaryRow@12ad64df, maxKey=org.apache.paimon.data.BinaryRow@2d67445f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-9.orc', minKey=org.apache.paimon.data.BinaryRow@a1fccfe1, maxKey=org.apache.paimon.data.BinaryRow@2e90249f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-10.orc', minKey=org.apache.paimon.data.BinaryRow@1646f5f, maxKey=org.apache.paimon.data.BinaryRow@fab41821}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-11.orc', minKey=org.apache.paimon.data.BinaryRow@8cec6b61, maxKey=org.apache.paimon.data.BinaryRow@df034df}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-12.orc', minKey=org.apache.paimon.data.BinaryRow@621adadf, maxKey=org.apache.paimon.data.BinaryRow@f8cd4661}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-13.orc', minKey=org.apache.paimon.data.BinaryRow@9bf5d2e1, maxKey=org.apache.paimon.data.BinaryRow@e9259221}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-14.orc', minKey=org.apache.paimon.data.BinaryRow@5faec55f, maxKey=org.apache.paimon.data.BinaryRow@1e49e1f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-15.orc', minKey=org.apache.paimon.data.BinaryRow@a5215be1, maxKey=org.apache.paimon.data.BinaryRow@cf1a84e1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-16.orc', minKey=org.apache.paimon.data.BinaryRow@b0b3fbe1, maxKey=org.apache.paimon.data.BinaryRow@de3cf2e1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-17.orc', minKey=org.apache.paimon.data.BinaryRow@bd4f52e1, maxKey=org.apache.paimon.data.BinaryRow@e0993a21}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-18.orc', minKey=org.apache.paimon.data.BinaryRow@a9a9b0a1, maxKey=org.apache.paimon.data.BinaryRow@f3f969f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-19.orc', minKey=org.apache.paimon.data.BinaryRow@9455da61, maxKey=org.apache.paimon.data.BinaryRow@16ebe55f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-20.orc', minKey=org.apache.paimon.data.BinaryRow@9bc8df61, maxKey=org.apache.paimon.data.BinaryRow@e4e533a1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-21.orc', minKey=org.apache.paimon.data.BinaryRow@aed529a1, maxKey=org.apache.paimon.data.BinaryRow@90a12f61}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-22.orc', minKey=org.apache.paimon.data.BinaryRow@8272c461, maxKey=org.apache.paimon.data.BinaryRow@1fd1f9df}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-23.orc', minKey=org.apache.paimon.data.BinaryRow@9808e921, maxKey=org.apache.paimon.data.BinaryRow@6a30709f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-24.orc', minKey=org.apache.paimon.data.BinaryRow@ddf851f, maxKey=org.apache.paimon.data.BinaryRow@541bf55f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-25.orc', minKey=org.apache.paimon.data.BinaryRow@34c78f9f, maxKey=org.apache.paimon.data.BinaryRow@1ce6755f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-26.orc', minKey=org.apache.paimon.data.BinaryRow@232f07df, maxKey=org.apache.paimon.data.BinaryRow@3ba7cd5f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-27.orc', minKey=org.apache.paimon.data.BinaryRow@4c77959f, maxKey=org.apache.paimon.data.BinaryRow@fb4c7721}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-28.orc', minKey=org.apache.paimon.data.BinaryRow@4918331f, maxKey=org.apache.paimon.data.BinaryRow@bd6b1e61}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-29.orc', minKey=org.apache.paimon.data.BinaryRow@ef7d8361, maxKey=org.apache.paimon.data.BinaryRow@31fcd8df}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-30.orc', minKey=org.apache.paimon.data.BinaryRow@7925549f, maxKey=org.apache.paimon.data.BinaryRow@f5a1a8a1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-31.orc', minKey=org.apache.paimon.data.BinaryRow@2b13d4df, maxKey=org.apache.paimon.data.BinaryRow@1c1cef9f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-32.orc', minKey=org.apache.paimon.data.BinaryRow@4dc6e19f, maxKey=org.apache.paimon.data.BinaryRow@1b4a4df}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-33.orc', minKey=org.apache.paimon.data.BinaryRow@ef731a21, maxKey=org.apache.paimon.data.BinaryRow@8f94f9f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-34.orc', minKey=org.apache.paimon.data.BinaryRow@29a1299f, maxKey=org.apache.paimon.data.BinaryRow@bc661461}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-35.orc', minKey=org.apache.paimon.data.BinaryRow@95ae1521, maxKey=org.apache.paimon.data.BinaryRow@29aeca1f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-36.orc', minKey=org.apache.paimon.data.BinaryRow@8f7249f, maxKey=org.apache.paimon.data.BinaryRow@3d68f49f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-37.orc', minKey=org.apache.paimon.data.BinaryRow@5981f41f, maxKey=org.apache.paimon.data.BinaryRow@9814e461}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-38.orc', minKey=org.apache.paimon.data.BinaryRow@33dcc71f, maxKey=org.apache.paimon.data.BinaryRow@4922299f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-39.orc', minKey=org.apache.paimon.data.BinaryRow@8e276d61, maxKey=org.apache.paimon.data.BinaryRow@6e7fe1f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-40.orc', minKey=org.apache.paimon.data.BinaryRow@39d502df, maxKey=org.apache.paimon.data.BinaryRow@6899b9f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-41.orc', minKey=org.apache.paimon.data.BinaryRow@48de24df, maxKey=org.apache.paimon.data.BinaryRow@5029809f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-42.orc', minKey=org.apache.paimon.data.BinaryRow@5e952e5f, maxKey=org.apache.paimon.data.BinaryRow@b079e5a1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-43.orc', minKey=org.apache.paimon.data.BinaryRow@b9a089a1, maxKey=org.apache.paimon.data.BinaryRow@2d5fbdf}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-44.orc', minKey=org.apache.paimon.data.BinaryRow@9f822f61, maxKey=org.apache.paimon.data.BinaryRow@81047721}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-45.orc', minKey=org.apache.paimon.data.BinaryRow@9da7b821, maxKey=org.apache.paimon.data.BinaryRow@d1e2abe1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-46.orc', minKey=org.apache.paimon.data.BinaryRow@537aeedf, maxKey=org.apache.paimon.data.BinaryRow@d4c4fda1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-47.orc', minKey=org.apache.paimon.data.BinaryRow@da284621, maxKey=org.apache.paimon.data.BinaryRow@c481c861}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=34, level=5, fileName='data-0fb904b5-8d0a-4cd7-8516-85c2385d1ef4-0.orc', minKey=org.apache.paimon.data.BinaryRow@4b0e9a22, maxKey=org.apache.paimon.data.BinaryRow@ebe7e41e}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=34, level=5, fileName='data-0fb904b5-8d0a-4cd7-8516-85c2385d1ef4-1.orc', minKey=org.apache.paimon.data.BinaryRow@95f2715e, maxKey=org.apache.paimon.data.BinaryRow@27ea0362}

Changes are:
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=37, level=0, fileName='data-19fd8e26-eeb3-4d61-87c8-87d8119d9aa7-233.orc', minKey=org.apache.paimon.data.BinaryRow@eefbdc9b, maxKey=org.apache.paimon.data.BinaryRow@a37b32db}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=37, level=0, fileName='data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc', minKey=org.apache.paimon.data.BinaryRow@5453f965, maxKey=org.apache.paimon.data.BinaryRow@3ec1f2e5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=37, level=1, fileName='data-c009c914-0ad6-415a-8f87-62fa7faef4ef-114.orc', minKey=org.apache.paimon.data.BinaryRow@6d4c825, maxKey=org.apache.paimon.data.BinaryRow@37d4abe5}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=37, level=1, fileName='data-11c0a96a-3138-427e-aeda-73868bd84d71-0.orc', minKey=org.apache.paimon.data.BinaryRow@5453f965, maxKey=org.apache.paimon.data.BinaryRow@3ec1f2e5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=42, level=0, fileName='data-341299c5-6126-4e97-90d7-f53532a02f7a-1421.orc', minKey=org.apache.paimon.data.BinaryRow@7c54a9ea, maxKey=org.apache.paimon.data.BinaryRow@dbbcd6a}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=42, level=0, fileName='data-87e8fabf-b9bd-4e84-a509-d31e30d2e379-1.orc', minKey=org.apache.paimon.data.BinaryRow@15072aea, maxKey=org.apache.paimon.data.BinaryRow@6b71422a}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=42, level=1, fileName='data-942ebe35-e4f3-4df2-a4b8-82e60394c9c2-698.orc', minKey=org.apache.paimon.data.BinaryRow@da7c53d6, maxKey=org.apache.paimon.data.BinaryRow@12a2afea}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=42, level=1, fileName='data-9f918459-4fc8-4991-8783-c709e7c80c8c-0.orc', minKey=org.apache.paimon.data.BinaryRow@15072aea, maxKey=org.apache.paimon.data.BinaryRow@6b71422a}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=43, level=0, fileName='data-64aa8e97-b138-4da7-8d40-2fb6887b954f-1427.orc', minKey=org.apache.paimon.data.BinaryRow@7f74906b, maxKey=org.apache.paimon.data.BinaryRow@d519ebd5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=43, level=1, fileName='data-d517bba2-f709-4432-be95-91bfaf04143d-702.orc', minKey=org.apache.paimon.data.BinaryRow@c626cc15, maxKey=org.apache.paimon.data.BinaryRow@44375d6b}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=43, level=0, fileName='data-29d21725-040e-438d-923d-77b708dc5d0b-1.orc', minKey=org.apache.paimon.data.BinaryRow@6e9da5ab, maxKey=org.apache.paimon.data.BinaryRow@5780d4ab}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=43, level=1, fileName='data-f7c6791a-25c1-4ef2-97fc-bd06c8b8e5a7-0.orc', minKey=org.apache.paimon.data.BinaryRow@6e9da5ab, maxKey=org.apache.paimon.data.BinaryRow@44375d6b}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=1, fileName='data-a44e51a9-c1a8-4e4f-9ff5-d8f78c8b0f1a-673.orc', minKey=org.apache.paimon.data.BinaryRow@25ef0ff4, maxKey=org.apache.paimon.data.BinaryRow@279e9334}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=0, fileName='data-f15d1be7-bed5-4540-9f85-812bb9f941b7-1431.orc', minKey=org.apache.paimon.data.BinaryRow@d73964cc, maxKey=org.apache.paimon.data.BinaryRow@495888b4}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=2, fileName='data-a44e51a9-c1a8-4e4f-9ff5-d8f78c8b0f1a-672.orc', minKey=org.apache.paimon.data.BinaryRow@43fad634, maxKey=org.apache.paimon.data.BinaryRow@6ab4c0f4}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=0, fileName='data-33ba253a-ac38-45ef-a93d-83842123daa6-1.orc', minKey=org.apache.paimon.data.BinaryRow@65a0a9b4, maxKey=org.apache.paimon.data.BinaryRow@981f4f4}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=2, fileName='data-ab8e0631-56db-46dc-802f-3c649138cd13-0.orc', minKey=org.apache.paimon.data.BinaryRow@65a0a9b4, maxKey=org.apache.paimon.data.BinaryRow@981f4f4}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=0, fileName='data-4152f05b-72af-49ae-9033-4d9390577a33-1427.orc', minKey=org.apache.paimon.data.BinaryRow@b323c58, maxKey=org.apache.paimon.data.BinaryRow@164ce58}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=1, fileName='data-52079d7c-b4b2-4779-84ae-72ee3f3e5b98-676.orc', minKey=org.apache.paimon.data.BinaryRow@51616ed8, maxKey=org.apache.paimon.data.BinaryRow@39b67258}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=2, fileName='data-52079d7c-b4b2-4779-84ae-72ee3f3e5b98-661.orc', minKey=org.apache.paimon.data.BinaryRow@28c68a18, maxKey=org.apache.paimon.data.BinaryRow@657e8f98}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=0, fileName='data-5c580eda-dc07-41cd-a9a8-3a1df276e8ac-1.orc', minKey=org.apache.paimon.data.BinaryRow@3e2f7f98, maxKey=org.apache.paimon.data.BinaryRow@bcd1a028}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=2, fileName='data-049815d2-f797-483e-aaef-d7ebf9e2568f-0.orc', minKey=org.apache.paimon.data.BinaryRow@3e2f7f98, maxKey=org.apache.paimon.data.BinaryRow@bcd1a028}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=0, fileName='data-6477a9c0-2b9f-4e1a-8b75-75cfb03a77ee-1429.orc', minKey=org.apache.paimon.data.BinaryRow@6aadea33, maxKey=org.apache.paimon.data.BinaryRow@55398af3}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=0, fileName='data-53f10294-3b79-4ceb-a337-a7f10026be1d-1.orc', minKey=org.apache.paimon.data.BinaryRow@927c29cd, maxKey=org.apache.paimon.data.BinaryRow@a2cd214d}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=1, fileName='data-b51ebdeb-4cef-4b10-86fc-d70fcc282642-679.orc', minKey=org.apache.paimon.data.BinaryRow@31ea51b3, maxKey=org.apache.paimon.data.BinaryRow@a0c8840d}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=3, fileName='data-b51ebdeb-4cef-4b10-86fc-d70fcc282642-223.orc', minKey=org.apache.paimon.data.BinaryRow@13c39fb3, maxKey=org.apache.paimon.data.BinaryRow@3a0e2b33}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=2, fileName='data-b51ebdeb-4cef-4b10-86fc-d70fcc282642-446.orc', minKey=org.apache.paimon.data.BinaryRow@a067e44d, maxKey=org.apache.paimon.data.BinaryRow@5d233273}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=4, fileName='data-b51ebdeb-4cef-4b10-86fc-d70fcc282642-0.orc', minKey=org.apache.paimon.data.BinaryRow@c2af094d, maxKey=org.apache.paimon.data.BinaryRow@fae7a70d}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=4, fileName='data-72a030b2-f604-4f8e-b58a-d296a895399b-0.orc', minKey=org.apache.paimon.data.BinaryRow@31ea51b3, maxKey=org.apache.paimon.data.BinaryRow@3a0e2b33}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=0, fileName='data-d66381fe-74ae-485f-bace-d6eb6ae4c010-1431.orc', minKey=org.apache.paimon.data.BinaryRow@9a15dacb, maxKey=org.apache.paimon.data.BinaryRow@411a2ef5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=1, fileName='data-bc06f732-a1db-487c-90fc-2923f718c404-675.orc', minKey=org.apache.paimon.data.BinaryRow@b6f5960b, maxKey=org.apache.paimon.data.BinaryRow@258b4935}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=2, fileName='data-bc06f732-a1db-487c-90fc-2923f718c404-569.orc', minKey=org.apache.paimon.data.BinaryRow@a1d6f20b, maxKey=org.apache.paimon.data.BinaryRow@e83b864b}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=0, fileName='data-dbb67efb-99cd-461e-9a10-206c2f311a25-1.orc', minKey=org.apache.paimon.data.BinaryRow@4cddbdb5, maxKey=org.apache.paimon.data.BinaryRow@507e4475}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=2, fileName='data-760fafac-0a11-4420-98cf-4b409c19691a-0.orc', minKey=org.apache.paimon.data.BinaryRow@4cddbdb5, maxKey=org.apache.paimon.data.BinaryRow@507e4475}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=0, fileName='data-39e993b3-3de3-4fd5-a2da-a08a83bb27cf-1433.orc', minKey=org.apache.paimon.data.BinaryRow@6324007c, maxKey=org.apache.paimon.data.BinaryRow@50739c3c}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=3, fileName='data-1d633c4e-7ba7-45a0-a800-5dba80a12a79-672.orc', minKey=org.apache.paimon.data.BinaryRow@a73c1984, maxKey=org.apache.paimon.data.BinaryRow@1621977c}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=2, fileName='data-1d633c4e-7ba7-45a0-a800-5dba80a12a79-678.orc', minKey=org.apache.paimon.data.BinaryRow@1282917c, maxKey=org.apache.paimon.data.BinaryRow@726d57bc}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=0, fileName='data-b3b8adb4-320f-4576-b90f-a154393b24fc-1.orc', minKey=org.apache.paimon.data.BinaryRow@a0cca2c4, maxKey=org.apache.paimon.data.BinaryRow@7ac72afc}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=3, fileName='data-ac05813a-32bf-49ee-aef1-9ae6ad7592ce-0.orc', minKey=org.apache.paimon.data.BinaryRow@a0cca2c4, maxKey=org.apache.paimon.data.BinaryRow@7ac72afc}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=48, level=0, fileName='data-8abde626-cfa9-48e7-ab18-03dcc0faabec-1431.orc', minKey=org.apache.paimon.data.BinaryRow@9d0206d0, maxKey=org.apache.paimon.data.BinaryRow@f0802b10}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=48, level=0, fileName='data-b2edcd3a-6358-4f42-9bce-0863e3c66466-1.orc', minKey=org.apache.paimon.data.BinaryRow@795b9270, maxKey=org.apache.paimon.data.BinaryRow@39b153f0}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=48, level=1, fileName='data-2121369d-785c-4282-bfa6-a64447ee5c29-705.orc', minKey=org.apache.paimon.data.BinaryRow@e97b3ed0, maxKey=org.apache.paimon.data.BinaryRow@a5d6e150}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=48, level=1, fileName='data-c758d136-64ea-421c-ae62-518259b00a2e-0.orc', minKey=org.apache.paimon.data.BinaryRow@e97b3ed0, maxKey=org.apache.paimon.data.BinaryRow@39b153f0}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=61, level=0, fileName='data-ce11c3a1-89d8-4946-a955-273bf1b0f1c9-1431.orc', minKey=org.apache.paimon.data.BinaryRow@4f43a9bd, maxKey=org.apache.paimon.data.BinaryRow@45e3067d}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=61, level=1, fileName='data-9b764057-5a10-4e6d-8a24-4290456836b8-714.orc', minKey=org.apache.paimon.data.BinaryRow@3a8d22bd, maxKey=org.apache.paimon.data.BinaryRow@c3b2c3c3}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=61, level=0, fileName='data-39c46c95-2498-447c-82f7-105f53115450-1.orc', minKey=org.apache.paimon.data.BinaryRow@964dd503, maxKey=org.apache.paimon.data.BinaryRow@dbd9d443}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=61, level=1, fileName='data-8a857c00-90f0-43a0-9167-ac41e0b7a3d4-0.orc', minKey=org.apache.paimon.data.BinaryRow@3a8d22bd, maxKey=org.apache.paimon.data.BinaryRow@c3b2c3c3}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=59, level=0, fileName='data-0b85af4f-7c88-4c84-8faa-38b5e87144d3-1431.orc', minKey=org.apache.paimon.data.BinaryRow@4407dbbb, maxKey=org.apache.paimon.data.BinaryRow@c4a77bc5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=59, level=0, fileName='data-e33879f0-c44d-4315-bb1f-c00cddfa12e6-1.orc', minKey=org.apache.paimon.data.BinaryRow@16f83d3b, maxKey=org.apache.paimon.data.BinaryRow@5ae6aefb}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=59, level=2, fileName='data-4c413d2d-7df2-4a31-a605-25900daf2ba1-442.orc', minKey=org.apache.paimon.data.BinaryRow@96c819c5, maxKey=org.apache.paimon.data.BinaryRow@1555d8fb}

The entry list above are not fully displayed, please refer to taskmanager.log for more information.
at org.apache.paimon.operation.FileStoreCommitImpl.createConflictException(FileStoreCommitImpl.java:1138) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:1010) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:990) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to delete file data-19fd8e26-eeb3-4d61-87c8-87d8119d9aa7-233.orc which is not previously added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.assertNoDelete(FileEntry.java:181) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:1007) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
... 20 more
2025-08-23 08:07:08,016 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 95c2d839e2e53a32f408c6e800c6df67
2025-08-23 08:07:08,021 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - 1 tasks will be restarted to recover the failed task 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_0.
2025-08-23 08:07:08,021 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RUNNING to RESTARTING.
2025-08-23 08:07:13,030 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RESTARTING to RUNNING.
2025-08-23 08:07:13,032 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from CREATED to SCHEDULED.
2025-08-23 08:07:13,033 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 95c2d839e2e53a32f408c6e800c6df67: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}]
2025-08-23 08:07:13,033 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from SCHEDULED to DEPLOYING.
2025-08-23 08:07:13,033 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (attempt #1) with attempt id 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1 and vertex id 250bb1b6a5242fdebff09c57fe4d1696_0 to container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473) with allocation id 2691f70b7c3b3db638d93c0aede41d20
2025-08-23 08:07:13,049 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from DEPLOYING to INITIALIZING.
2025-08-23 08:07:14,575 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from INITIALIZING to RUNNING.
2025-08-23 08:07:15,318 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from RUNNING to FAILED on container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473).
java.lang.RuntimeException: Cannot read manifest entries from changed partitions.
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:984) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to add file {org.apache.paimon.data.BinaryRow@9c67b85d, 37, 0, data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc} which is already added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:134) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:114) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readAndMergeFileEntries(AbstractFileStoreScan.java:397) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readSimpleEntries(AbstractFileStoreScan.java:259) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:982) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
... 20 more
2025-08-23 08:07:15,320 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 95c2d839e2e53a32f408c6e800c6df67
2025-08-23 08:07:15,321 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - 1 tasks will be restarted to recover the failed task 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1.
2025-08-23 08:07:15,321 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RUNNING to RESTARTING.
2025-08-23 08:07:20,322 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RESTARTING to RUNNING.
2025-08-23 08:07:20,322 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from CREATED to SCHEDULED.
2025-08-23 08:07:20,323 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 95c2d839e2e53a32f408c6e800c6df67: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}]
2025-08-23 08:07:20,323 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from SCHEDULED to DEPLOYING.
2025-08-23 08:07:20,323 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (attempt #2) with attempt id 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2 and vertex id 250bb1b6a5242fdebff09c57fe4d1696_0 to container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473) with allocation id 2691f70b7c3b3db638d93c0aede41d20
2025-08-23 08:07:20,335 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from DEPLOYING to INITIALIZING.
2025-08-23 08:07:21,252 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from INITIALIZING to RUNNING.
2025-08-23 08:07:22,321 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from RUNNING to FAILED on container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473).
java.lang.RuntimeException: Cannot read manifest entries from changed partitions.
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:984) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to add file {org.apache.paimon.data.BinaryRow@9c67b85d, 37, 0, data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc} which is already added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:134) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:114) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readAndMergeFileEntries(AbstractFileStoreScan.java:397) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readSimpleEntries(AbstractFileStoreScan.java:259) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:982) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
... 20 more
2025-08-23 08:07:22,322 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 95c2d839e2e53a32f408c6e800c6df67
2025-08-23 08:07:22,323 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - 1 tasks will be restarted to recover the failed task 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2.
2025-08-23 08:07:22,323 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RUNNING to RESTARTING.
2025-08-23 08:07:27,324 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RESTARTING to RUNNING.
2025-08-23 08:07:27,324 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from CREATED to SCHEDULED.
2025-08-23 08:07:27,325 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from SCHEDULED to DEPLOYING.
2025-08-23 08:07:27,324 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 95c2d839e2e53a32f408c6e800c6df67: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}]
2025-08-23 08:07:27,325 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (attempt #3) with attempt id 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3 and vertex id 250bb1b6a5242fdebff09c57fe4d1696_0 to container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473) with allocation id 2691f70b7c3b3db638d93c0aede41d20
2025-08-23 08:07:27,338 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from DEPLOYING to INITIALIZING.
2025-08-23 08:07:28,229 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from INITIALIZING to RUNNING.
2025-08-23 08:07:29,558 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from RUNNING to FAILED on container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473).
java.lang.RuntimeException: Cannot read manifest entries from changed partitions.
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:984) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to add file {org.apache.paimon.data.BinaryRow@9c67b85d, 37, 0, data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc} which is already added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:134) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:114) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readAndMergeFileEntries(AbstractFileStoreScan.java:397) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readSimpleEntries(AbstractFileStoreScan.java:259) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:982) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
... 20 more
2025-08-23 08:07:29,560 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 95c2d839e2e53a32f408c6e800c6df67
2025-08-23 08:07:29,561 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RUNNING to FAILING.
org.apache.flink.runtime.JobException: Recovery is suppressed by FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=3, backoffTimeMS=5000)
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:139) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:83) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.DefaultScheduler.recordTaskFailure(DefaultScheduler.java:258) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:249) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.DefaultScheduler.onTaskFailed(DefaultScheduler.java:242) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:748) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:725) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:80) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:479) ~[flink-dist-1.17.2.jar:1.17.2]
at sun.reflect.GeneratedMethodAccessor79.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_311]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_311]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:309) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:307) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:222) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:84) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:168) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.ActorCell.invoke(ActorCell.scala:547) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_311]
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1067) [?:1.8.0_311]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1703) [?:1.8.0_311]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:172) [?:1.8.0_311]
Caused by: java.lang.RuntimeException: Cannot read manifest entries from changed partitions.
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:984) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to add file {org.apache.paimon.data.BinaryRow@9c67b85d, 37, 0, data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc} which is already added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:134) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:114) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readAndMergeFileEntries(AbstractFileStoreScan.java:397) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readSimpleEntries(AbstractFileStoreScan.java:259) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:982) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
2025-08-23 08:07:29,566 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 4460de05e51badec3bb31e202627eb02_cbc357ccb763df2852fee8c4fc7d55f2_0_0.
2025-08-23 08:07:29,567 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 4460de05e51badec3bb31e202627eb02_cbc357ccb763df2852fee8c4fc7d55f2_1_0.

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Paimon 0.8 table definition and the two Flink 1.17.2 SQL jobs: the streaming INSERT and batch DELETE. Reproduce the Global Committer failure reporting “File deletion conflicts detected” while both jobs write to the same table, then determine the conditions that cause the conflict and verify that the jobs can commit without the error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
data-engineering, databases, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.