apache / apache/rocketmq-flink
[Feature][RIP-74] Rocketmq-flink dynamic load balancing
- Dominant language
- Java
- Stars
- 174
- Forks
- 104
- PR merge metrics
- No merged PRs in 30d
Description
In the original version, rocketmq-flink's queue allocation algorithm could not be averaged and could exacerbate unevenness when reduced. When the online environment experiences frequent scale-down, the situation becomes worse, and if you want to alleviate this situation, you can only restart the job, but cause large pauses. Therefore, we need to implement runtime dynamic allocation of queues.
RIP-docs: https://shimo.im/docs/RKAWMOEYgXfM88q8
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked RIP-docs design document, then trace the existing RocketMQ-Flink queue allocation logic in the source. Define how queue ownership changes during runtime scale-down and verify that rebalancing completes without requiring a job restart or causing large pauses.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100