apache / apache/pulsar

Regression in the behavior of subscribe()

Open
#11,289 3 comments 0 reactions 0 assignees View on GitHub
lifecycle/stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
If a topic is created, and a subscribe() is done on a ConsumerBuilder using that topic, the subscribe hangs.
This is a regression.

The unit-test which shows this issue runs to completion on Pulsar up to commit-id 555042a45d9692148c302fbdedbb47f1f2572a48
But it stalls starting from commit-id 794aa20d9f2a4e668cc36465362d22e042e6e536

**To Reproduce**
The issue can be reproduced by executing the unit test in the commit below.
[See https://github.com/apache/pulsar/pull/11290 and run
mvn test -Dtest=org.apache.pulsar.broker.resourcegroup.StallSubscribeTest -pl pulsar-broker]

On a Pulsar repo sync-ed to commit-id 555042a45d9692148c302fbdedbb47f1f2572a48:
% mvn test -Dtest=org.apache.pulsar.broker.resourcegroup.StallSubscribeTest -pl pulsar-broker
[...]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.pulsar.broker.resourcegroup.StallSubscribeTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.226 s - in org.apache.pulsar.broker.resourcegroup.StallSubscribeTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 20.512 s
[INFO] Finished at: 2021-07-12T12:35:38-07:00
[INFO] ------------------------------------------------------------------------
%

With the repo sync-ed to 794aa20d9f2a4e668cc36465362d22e042e6e536 (or later):
% mvn test -Dtest=org.apache.pulsar.broker.resourcegroup.StallSubscribeTest -pl pulsar-broker
[...]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.pulsar.broker.resourcegroup.StallSubscribeTest
[ERROR] Tests run: 4, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 129.393 s <<< FAILURE! - in org.apache.pulsar.broker.resourcegroup.StallSubscribeTest
[ERROR] testProduceConsumeUsageOnRG(org.apache.pulsar.broker.resourcegroup.StallSubscribeTest) Time elapsed: 60.016 s <<< FAILURE!
java.lang.AssertionError: Got exception while building consumer: ex={} expected [true] but found [false]
at org.testng.Assert.fail(Assert.java:99)
at org.testng.Assert.failNotEquals(Assert.java:1037)
at org.testng.Assert.assertTrue(Assert.java:45)
at org.apache.pulsar.broker.resourcegroup.StallSubscribeTest.testProduceConsumeUsageOnRG(StallSubscribeTest.java:72)
at org.apache.pulsar.broker.resourcegroup.StallSubscribeTest.testProduceConsumeUsageOnRG(StallSubscribeTest.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] org.apache.pulsar.broker.resourcegroup.StallSubscribeTest.testProduceConsumeUsageOnRG(org.apache.pulsar.broker.resourcegroup.StallSubscribeTest)
[INFO] Run 1: PASS
[ERROR] Run 2: StallSubscribeTest.testProduceConsumeUsageOnRG:53->testProduceConsumeUsageOnRG:72 Got exception while building consumer: ex={} expected [true] but found [false]
[INFO]
[INFO]
[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 2
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:25 min
[INFO] Finished at: 2021-07-12T12:41:17-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M3:test (default-test) on project pulsar-broker: There are test failures.
[ERROR]

**Expected behavior**
The unit-test is expected to complete.

**Workaround**
Immediate workaround: Don't create the topic; do the subscribe without topic creation (not sure if it will result in some other issue downstream).

**Desktop (please complete the following information):**
% uname -a
Darwin C02D2553MD6R 19.6.0 Darwin Kernel Version 19.6.0: Thu May 6 00:48:39 PDT 2021; root:xnu-6153.141.33~1/RELEASE_X86_64 x86_64
%

**Additional context**
This issue is resulting in the existing UT RGUsageMTAggrWaitForAllMesgs timing out on my desktop.
[mvn test -Dtest=org.apache.pulsar.broker.resourcegroup.RGUsageMTAggrWaitForAllMesgs -pl pulsar-broker]

Contributor guide

Open the contributing guide

Research direction

Start with pulsar-broker's org.apache.pulsar.broker.resourcegroup.StallSubscribeTest and run it with Maven using the command in the issue. Compare behavior between commits 555042a45d9692148c302fbdedbb47f1f2572a48 and 794aa20d9f2a4e668cc36465362d22e042e6e536, focusing on subscribe after topic creation. Done means the test completes successfully without the consumer-build timeout or failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.