[Bug] Topics appended unlimitedly when subscribing with topics pattern
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
2.10.2.4
### Minimal reproduce step
Due to consumers keep subscribing to their own retries and dead letter messages, Topic is infinitely appended (`allowAutoTopicCreation = true`)
The code used to reproduce:
```java
public static void main(String[] args) throws IOException {
AuthenticationBasic auth = new AuthenticationBasic();
auth.configure("{\"userId\":\"superuser\",\"password\":\"admin\"}");
String serviceUrl = "pulsar://127.0.0.1:6650";
String topicPattern = "persistent://public/default/ubw.*";
PulsarClient pulsarClient = PulsarClient.builder().serviceUrl(serviceUrl).authentication(auth).build();
var consumerBuilder = pulsarClient.newConsumer(Schema.BYTES).topicsPattern(topicPattern).subscriptionName("ubw-test")
.subscriptionType(SubscriptionType.Shared)
.enableRetry(true)
.deadLetterPolicy(DeadLetterPolicy.builder()
.maxRedeliverCount(3)
.build())
.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
.messageListener((consumer, msg) -> {
System.out.println("=======================================");
System.out.println("msg = " + new String(msg.getData()));
System.out.println("topic = " + msg.getTopicName());
System.out.println("key = " + msg.getKey());
System.out.println("properties = " + msg.getProperties());
try {
consumer.reconsumeLater(msg, 1, TimeUnit.SECONDS);
} catch (PulsarClientException e) {
throw new RuntimeException(e);
}
}).patternAutoDiscoveryPeriod(1, TimeUnit.SECONDS);
consumerBuilder.subscribe();
System.in.read();
}
```
### What did you expect to see?
This is a bit strange, I think it should automatically acknowledge when the dead letter message is consumed?
This problem will eventually cause Pulsar to throw an exception:
`Got exception org.apache.pulsar.shade.io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 5253120: 5263796 - discarded`
### What did you see instead?
Topic is constantly appended, reaching a terrifying length
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Start with the provided Java reproduction and trace topic-pattern discovery alongside retry and dead-letter handling. Determine why generated retry or dead-letter topics are repeatedly included; done means the topic list no longer grows indefinitely and the reported frame-length exception is avoided.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100