apache / apache/helix

Request for Fair State Transition Scheduling Across Multiple Resources

Open
#3,021 12 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
504
Forks
255
Avg merge
17d 47m
Merged PRs (30d)
2

Description

**Is your feature request related to a problem? Please describe.**
We use CUSTOMIZED mode and today there does not seem to be any fairness in terms of how the Helix state transition messages are sent out to the cluster across different resources.

For example, say I have two resources, `TableA` and `TableB` that share the same instances. If I update `TableA` with a very large number of IdealState changes (say 10K), then updates to `TableB` land up getting queued behind `TableA` and no progress is made until `TableA`'s state transitions complete.

This leads to starvation for `TableB`. Ideally resources should not affect each other in a Helix cluster.

**Describe the solution you'd like**
Ideally the Helix state transitions should be scheduled fairly across resources in a cluster to prevent the above starvation scenario.

**Additional context**
I know WAGED solves some of these problems, but moving to WAGED is not an option for us. Are ther any other solutions that can be leveraged?

cc @junkaixue @zpinto

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how state-transition messages are scheduled in CUSTOMIZED mode and reproduce the TableA/TableB starvation scenario with a large batch of IdealState changes. Done should be a decided and demonstrated scheduling approach that lets both resources make progress fairly without requiring WAGED.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.