Subsctiption per topic limit reached
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 142
Description
**Describe the bug**
maxSubscriptionsPerTopic is set to 0 and yet all of our consumers are throwing:
{"timestamp":"2021-06-29T06:31:23.683Z","logger":"org.apache.pulsar.client.impl.MultiTopicsConsumerImpl","thread":"pulsar-external-listener-3-1","level":"WARN","stacktrace":"java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException$ConsumerBusyException: Subscription reached max consumers limit\n\tat java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)\n\tat java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)\n\tat java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:777)\n\tat java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)\n\tat java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2152)\n\tat org.apache.pulsar.client.impl.ClientCnx.handleError(ClientCnx.java:633)\n\tat org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:179)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat org.apache.pulsar.shade.io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1486)\n\tat org.apache.pulsar.shade.io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1247)\n\tat org.apache.pulsar.shade.io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)\n\tat org.apache.pulsar.shade.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)\n\tat org.apache.pulsar.shade.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475)\n\tat org.apache.pulsar.shade.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)\n\tat org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)\n\tat org.apache.pulsar.shade.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat java.base/java.lang.Thread.run(Thread.java:832)\nCaused by: org.apache.pulsar.client.api.PulsarClientException$ConsumerBusyException: Subscription reached max consumers limit\n\tat org.apache.pulsar.client.impl.ClientCnx.getPulsarClientException(ClientCnx.java:966)\n\t... 30 common frames omitted\n","msg":"Failed subscription for createPartitionedConsumer: internal/emsaas/enriched-asset-event 3, e:{}"}
**To Reproduce**
Steps to reproduce the behavior:
1. Only way to reproduce is just adding more consumers until the error pops up
**Expected behavior**
As per documentation, "maxSubscriptionsPerTopic" set to 0 should disable the check completely
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- EKS helm chart deployed pulsar
Contributor guide
Research direction
Start by tracing the reported stack through MultiTopicsConsumerImpl, ClientCnx, and the createPartitionedConsumer path, then locate how maxSubscriptionsPerTopic is enforced. Reproduce the issue by adding consumers until the limit error appears. Done means setting maxSubscriptionsPerTopic to 0 disables the check as documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100