[Bug] Sequence group may cause record in wrong state.
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
If my records come disorderly, -D record may have been compact out of table. Then +I (which sequence number is smaller then -D record) may survive.
StreamWriteBuilder builder = getTable().newStreamWriteBuilder();
StreamTableWrite write = builder.newWrite();
write.withIOManager(new IOManagerImpl("~/temp/"));
StreamTableCommit commit = builder.newCommit();
// +I[2025-04-01, 1, type1, 29, 5, name2, 29, 5, name2, null, 9223372036854775802]
GenericRow genericRow = GenericRow.of(
1L,
2L,
2L);
GenericRow genericRow2 = GenericRow.ofKind(RowKind.DELETE,
1L,
null,
Long.MAX_VALUE - 1);
GenericRow genericRow3 = GenericRow.of(
1L,
1L,
1L);
write.write(genericRow);
commit.commit(1L, write.prepareCommit(true, 1L));
write.write(genericRow2);
commit.commit(2L, write.prepareCommit(true, 2L));
readData();
write.write(genericRow3);
commit.commit(3L, write.prepareCommit(true, 3L));
Record should be 1,null,Long.MAX_VALUE-1
### Compute Engine
Flink
### Minimal reproduce step
above
### What doesn't meet your expectations?
--
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the provided minimal reproduction and trace the sequence-group behavior through StreamWriteBuilder, StreamTableWrite, StreamTableCommit, and readData. Verify the disorderly +I and -D sequence described in the issue, then confirm that the final record is 1,null,Long.MAX_VALUE-1.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100