apache / apache/pulsar

[Bug] Topic compaction not working as expected and leading to ack holes on the subscription while read compacted true

Open
#21,986 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Version

2.10.4

### Minimal reproduce step

Create a sample producer which produces key based messages .like
k1 value1
k2 value2
k1 ""
k2 ""

create the sample consumer where trigger compaction and read with read compacted true .

value read will be k1 ""
k2 ""

but when we see the internal stats ,observe the ack holes .

### What did you expect to see?

ack holes should not be their on the subscriptions .

### What did you see instead?

ack holes is seen on the subscription .

### Anything else?

[compaction.txt](https://github.com/apache/pulsar/files/14084220/compaction.txt)
attached the sample produce and consume along with internal stats and stats for the topic .

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with the attached compaction.txt reproduction and its internal topic and subscription statistics. Reproduce the key-based messages with compaction enabled and read compacted set to true, then trace the compaction and acknowledgement behavior. Done means the compacted subscription returns the expected tombstone values without reporting acknowledgement holes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.