apache / apache/pulsar

[Bug] msgbacklog grows infinitely after sending a certain amount of messages

Open
#23,606 6 comments 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Read release policy

- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

### Version

3.0.1

### Minimal reproduce step

1. Send 1000 messages and stop the producer process
2. msgbacklog has 1000 messages
3. Consume 1,000 messages and stop the consumer process
4. After the consumer stops, msgbacklog is 0

### What did you expect to see?

msgbacklog sees messages that are equal to or less than the number of messages I sent

### What did you see instead?

The producer stops sending after sending a batch of messages. When subscribing to these states, it is found that the msgbacklog value has been growing continuously.

### Anything else?
**This problem occurs when I set replicateSubscriptionState to true in the consumer!!!**

After both the producer and the consumer are stopped, the msgbacklog has been growing slowly, and no additional messages will be consumed when the consumer is restarted.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue on Pulsar 3.0.1 with replicateSubscriptionState set to true, following the producer and consumer steps in the report. Investigate why msgbacklog keeps growing after both processes stop; it is done when backlog does not exceed the sent messages and restarting the consumer can consume the expected messages.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.