apache / apache/pulsar

[Bug] Dynamically expanding partitions leads to a continuous increase in the number of consumers

Open
#23,936 1 comment 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Read release policy

- [x] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

### Version

java 1.8
pulsar client 2.7.2
pulsar broker 2.7.2

### Minimal reproduce step

The broker notified the consumer to go offline while triggering the execution of the dynamic expansion partition task. Reconnected consumers cannot be restored

### What did you expect to see?

No need to exclude consumers, during the process of reconnecting consumers

### What did you see instead?

consumers.entrySet().removeIf(e -> TopicName.get(e.getKey()).getPartitionedTopicName().equals(topicName) && !e.getValue().isConnected());

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with Pulsar client and broker 2.7.2 using the dynamic partition expansion scenario. Trace the consumer cleanup around the reported consumers.entrySet().removeIf expression and verify that reconnecting consumers are restored without a continuing increase in consumer count.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.