apache / apache/pulsar

[improvement][broker]Add replicate all subscription state switch on the broker side.

Open
#23,768 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

In the current version of Pulsar, when performing cluster data synchronization, in order to continue consuming the consumption message location recorded in the source cluster after switching from the source cluster to the target cluster, it is necessary to configure the parameter `repliceSubscriptionState=true` on the client side to replicate the subscribed consumption location information.
There may be some issues with this, such as when my cluster needs to be migrated, if there are dozens of business teams involved in this cluster, then I need to align the migration action with all business teams to ensure that everyone switches and changes at the same pace.
However, this difficulty is very high, as there are complex upstream and downstream dependencies in the data flow between different businesses. At the same time, the business needs to modify the code online in order to replicate the subscription status, which makes our migration efficiency heavily dependent on the business.
For this scenario, we expect to have a global control switch for subscription state replication on the server, which can be enabled to replicate all subscription state information in the cluster without the need for business code modification.

### Solution

Add a switch on the broker side to replicate all subscription states from the source cluster to the target cluster.

### Alternatives

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!
PR: https://github.com/apache/pulsar/pull/23769

Contributor guide

Open the contributing guide

Research direction

Review the existing client-side `repliceSubscriptionState=true` behavior and the broker-side replication flow described in the issue, then inspect PR #23769 for the work already in progress. Done means a broker-level switch can replicate all subscription state without business code changes, with appropriate coverage for the migration scenario.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.