Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improvement][broker]Add replicate all subscription state switch on the broker side. #23768

Open
2 tasks done
yyj8 opened this issue Dec 21, 2024 · 0 comments
Open
2 tasks done
Labels
type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages

Comments

@yyj8
Copy link
Contributor

yyj8 commented Dec 21, 2024

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

In the current version of Pulsar, when performing cluster data synchronization, in order to continue consuming the consumption message location recorded in the source cluster after switching from the source cluster to the target cluster, it is necessary to configure the parameter repliceSubscriptionState=true on the client side to replicate the subscribed consumption location information.
There may be some issues with this, such as when my cluster needs to be migrated, if there are dozens of business teams involved in this cluster, then I need to align the migration action with all business teams to ensure that everyone switches and changes at the same pace.
However, this difficulty is very high, as there are complex upstream and downstream dependencies in the data flow between different businesses. At the same time, the business needs to modify the code online in order to replicate the subscription status, which makes our migration efficiency heavily dependent on the business.
For this scenario, we expect to have a global control switch for subscription state replication on the server, which can be enabled to replicate all subscription state information in the cluster without the need for business code modification.

Solution

Add a switch on the broker side to replicate all subscription states from the source cluster to the target cluster.

Alternatives

No response

Anything else?

No response

Are you willing to submit a PR?

@yyj8 yyj8 added the type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages label Dec 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

No branches or pull requests

1 participant