[Bug] Query result duplicate primary key
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
0.7.0-incubating
### Compute Engine
Flink 1.18.0
### Minimal reproduce step
we have a flink job write data to paimon table,table options is:
+---------------------------+---------+
| key | value |
+---------------------------+---------+
| bucket | 8 |
| scan.remove-normalize | true |
| deduplicate.ignore-delete | true |
| changelog-producer | none |
| file.format | parquet |
+---------------------------+---------+
### What doesn't meet your expectations?
When job execute for some time, we use batch mode query table ,some of our query results duplicate primary key.

When we update paimon version to query this table,it also has duplicate primary key.
### Anything else?
I want to know what cause this problem. Is this problem caused by writer operator? Does a later version fix this issue?
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the duplicate-primary-key result with Paimon 0.7.0-incubating, Flink 1.18.0, batch mode, and the table options listed in the issue. Compare the result with a later Paimon version and trace whether the cause is in the writer or query path; done means identifying the responsible component and affected versions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100