apache / apache/pulsar

Subsctiption per topic limit reached

Open
#11,142 2 comments 0 reactions 0 assignees View on GitHub
lifecycle/stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 22h
Merged PRs (30d)
142

Description

**Describe the bug**
maxSubscriptionsPerTopic is set to 0 and yet all of our consumers are throwing:

{"timestamp":"2021-06-29T06:31:23.683Z","logger":"org.apache.pulsar.client.impl.MultiTopicsConsumerImpl","thread":"pulsar-external-listener-3-1","level":"WARN","stacktrace":"java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException$ConsumerBusyException: Subscription reached max consumers limit\n\tat java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)\n\tat java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)\n\tat java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:777)\n\tat java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)\n\tat java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2152)\n\tat org.apache.pulsar.client.impl.ClientCnx.handleError(ClientCnx.java:633)\n\tat org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:179)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat org.apache.pulsar.shade.io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1486)\n\tat org.apache.pulsar.shade.io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1247)\n\tat org.apache.pulsar.shade.io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)\n\tat org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)\n\tat org.apache.pulsar.shade.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)\n\tat org.apache.pulsar.shade.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475)\n\tat org.apache.pulsar.shade.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)\n\tat org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)\n\tat org.apache.pulsar.shade.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat java.base/java.lang.Thread.run(Thread.java:832)\nCaused by: org.apache.pulsar.client.api.PulsarClientException$ConsumerBusyException: Subscription reached max consumers limit\n\tat org.apache.pulsar.client.impl.ClientCnx.getPulsarClientException(ClientCnx.java:966)\n\t... 30 common frames omitted\n","msg":"Failed subscription for createPartitionedConsumer: internal/emsaas/enriched-asset-event 3, e:{}"}

**To Reproduce**
Steps to reproduce the behavior:
1. Only way to reproduce is just adding more consumers until the error pops up

**Expected behavior**
As per documentation, "maxSubscriptionsPerTopic" set to 0 should disable the check completely

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- EKS helm chart deployed pulsar

Contributor guide

Open the contributing guide

Research direction

Start by tracing the reported stack through MultiTopicsConsumerImpl, ClientCnx, and the createPartitionedConsumer path, then locate how maxSubscriptionsPerTopic is enforced. Reproduce the issue by adding consumers until the limit error appears. Done means setting maxSubscriptionsPerTopic to 0 disables the check as documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.