slow consumer will cause backlog quota exceeded
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
**Is your feature request related to a problem? Please describe.**
if we have two consumers in one subscription, one is slow(connection is ok, but only slow), the it will hold some unacked messages. use `produce exception` strategy
after a while, backlogsize will increase, and new producers will get
Could not get connection to broker: Cannot create producer on topic with
backlog quota exceeded -- Will try again xx
**Describe the solution you'd like**
maybe we can add
```
public void handleExceededBacklogQuota(PersistentTopic persistentTopic) {
TopicName topicName = TopicName.get(persistentTopic.getName());
BacklogQuota quota = getBacklogQuota(topicName);
log.info("Backlog quota exceeded for topic [{}]. Applying [{}] policy", persistentTopic.getName(),
quota.getPolicy());
switch (quota.getPolicy()) {
case consumer_backlog_eviction:
dropBacklog(persistentTopic, quota);
break;
case producer_exception:
case producer_request_hold:
disconnectProducers(persistentTopic);
break;
case resend unacked: // add new strategy
default:
break;
}
}
```
or. in the dispatcher, record the timeout.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
Contributor guide
Research direction
Start by reviewing the PersistentTopic backlog-quota handling and the dispatcher behavior mentioned in the issue. Compare the proposed handleExceededBacklogQuota flow with recording a timeout for unacked messages, then define how the new resend-unacked strategy should interact with existing quota policies. Done means the behavior for a slow consumer and new producers is specified and verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100