apache / apache/pulsar

Support user defined and pluggable consumer selector

Open
#13,347 16 comments 0 reactions 1 assignee Claimed by @gaozhangmin View on GitHub
lifecycle/stale Stale type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Is your enhancement request related to a problem? Please describe.**
When we try to Introduce pulsar to our existing system, compatibility is our first consideration. Firstly, our production system uses jump consistent hash algorithm to select consumers, but pulsar uses range consistent hash algorithm natively. It's impossible for us to guarantee that the same node can consume data with the same routing key from pulsar and our existing system simultaneously. Secondly, our system requires no change in consumer selecting rules when one or more consumers disconnect temporarily(service upgrade.etc). As our service is locally stateful and will be updated frequently, constantly route-jitter is harmful for us.

**Describe the solution you'd like**
Support user defined and pluggable consumer selector, so we can manage consumers' behavior by ourself when dispath messages.

**Describe alternatives you've considered**
We have implemented the same routing algorithm and consumer management strategy as our existing system in broker. But this implementation will cause great trouble for us to update the version of puslar regularly in the future.
@codelipenghui

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.