apache / apache/pulsar

[Bug] Topic compaction causes subscription backlog growth

Open
#23,245 5 comments 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Read release policy

- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

### Version

OSX 14.3.1 (23D60), JDK 17, Pulsar 3.3.1

### Minimal reproduce step

When last message sent to topic has no value and such topic is compacted then subscription that reads this compacted topic has messages in backlog. Please check the junit Test below. It fails but it should pass just fine.

```java
@Test
public void testCompactionWithEmptyValue() throws Exception {
String topic = "persistent://my-property/use/my-ns/test1";

Producer producer = pulsarClient.newProducer()
.topic(topic)
.enableBatching(false)
.messageRoutingMode(MessageRoutingMode.SinglePartition)
.create();

pulsarClient.newConsumer().topic(topic).subscriptionName("sub1").readCompacted(true).subscribe().close();

producer.newMessage().key("withValue").value("data".getBytes()).send();
producer.newMessage().key("emptyValue").send();

admin.topics().triggerCompaction(topic);
Awaitility.await().atMost(1, TimeUnit.MINUTES).untilAsserted(() -> Assert.assertEquals(admin.topics().compactionStatus(topic).status, Status.SUCCESS));

try (Consumer consumer = pulsarClient.newConsumer().topic(topic).subscriptionName("sub1")
.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
.subscriptionType(SubscriptionType.Exclusive)
.readCompacted(true).subscribe()) {
for (int i=0; i<2 ; i++) {
Message m = consumer.receive(2, TimeUnit.SECONDS);
if (m != null) {
consumer.acknowledgeCumulative(m);
}
}

Awaitility.await().untilAsserted(() ->
Assert.assertEquals(admin.topics().getStats(topic).getSubscriptions().get("sub1").getMsgBacklog(), 0));
}
}

```
### What did you expect to see?

No message in backlog of the subscription that reads compacted topic.

### What did you see instead?

Message backlog is not empty when subscription read compacted topic.

### Anything else?

I verified this behavior on both running Pulsar and unit test. If you change second message
`producer.newMessage().key("emptyValue").send()` to `producer.newMessage().key("emptyValue").value("data".getBytes()).send()`
The test passes.

The behavior should be the same regardless the presence of payload in last message on the topic.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with the provided testCompactionWithEmptyValue reproduction and run it against the Pulsar test suite. Trace topic compaction and readCompacted subscription backlog handling, then confirm the test passes and the subscription backlog reaches zero when the final message has no value.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.