apache / apache/pulsar

The reconsumeLater() method that uses retry queues builds a large number of producers

Open
#23,533 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

Java-SD dispatch "public void reconsumeLater(Message message, long delayTime, TimeUnit unit)" method builds retry queue producers for each consumer per partition number of the original topic, causing the retry queue producers to severely exceed the limit

### Solution

_No response_

### Alternatives

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by locating the Java-SD implementation of reconsumeLater(Message message, long delayTime, TimeUnit unit) and trace how retry queue producers are created for partitioned topics. Confirm the producer count no longer scales excessively with consumers and original-topic partitions, then add or run focused coverage for this behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.