apache / apache/pulsar

[2.8.1]In Geo-Replication, "Failover" mode subscriptions consume messages repeatedly.

Open
#14,817 10 comments 0 reactions 0 assignees View on GitHub
lifecycle/stale Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
I have Geo replicated Pulsar clusters: Cluster-A and Cluster-B. Consumer 1 subscribes to Cluster-A, and Consumer 2 subscribes to Cluster-B. Consumer 1 and Consumer 2 have the same subscription name and use the "Failover" mode.

When I start consumer 1 first, consume a message and ACK. Consumer 2 is started after 30 seconds, and it is found that consumer 2 has repeatedly consumed messages.

Both Consumer 1 and Consumer 2 have replicateSubscriptionState set to true.

Configurations related to subscription replication are default values. as follows:
![image](https://user-images.githubusercontent.com/41796520/159652869-f68b5295-a667-42b5-80c0-bcf11fcfaab0.png)

**To Reproduce**
1. Start consumer 1.
2. Start a producer to send messages to Cluster A.
3. See consumer 1 consume the message, then wait 30 seconds.
4. Start consumer 2.

**Expected behavior**
Consumer 2 should not repeatedly consume messages that Consumer 1 has acked.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: Ubuntu 20.04.3 LTS

**Additional context**
- Pulsar version: 2.8.1

Contributor guide

Open the contributing guide

Research direction

Reproduce the scenario with Geo-Replication, two consumers using the same subscription in Failover mode, and replicateSubscriptionState enabled. Start with the subscription replication configuration and the consumer failover behavior; done means Consumer 2 does not repeatedly consume messages already acknowledged by Consumer 1.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.