[Bug] Dynamically expanding partitions leads to a continuous increase in the number of consumers
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [x] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### Version
java 1.8
pulsar client 2.7.2
pulsar broker 2.7.2
### Minimal reproduce step
The broker notified the consumer to go offline while triggering the execution of the dynamic expansion partition task. Reconnected consumers cannot be restored
### What did you expect to see?
No need to exclude consumers, during the process of reconnecting consumers
### What did you see instead?
consumers.entrySet().removeIf(e -> TopicName.get(e.getKey()).getPartitionedTopicName().equals(topicName) && !e.getValue().isConnected());
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Start by reproducing the issue with Pulsar client and broker 2.7.2 using the dynamic partition expansion scenario. Trace the consumer cleanup around the reported consumers.entrySet().removeIf expression and verify that reconnecting consumers are restored without a continuing increase in consumer count.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100