Support configurable segment balancer strategy
Open
feature
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Currently pinot doesn't consider disk usage ratio when distribute segments and cause high skewed issue.
Refer to Druid, which can support configurable segment balancer.
https://druid.apache.org/docs/latest/configuration/index.html
druid.coordinator.balancer.strategy
Contributor guide
Research direction
Start with the issue description and the linked Druid balancer configuration, especially druid.coordinator.balancer.strategy. Define how Pinot should make segment balancing configurable and account for disk usage ratio; done should include a documented strategy and validation that balancing reduces skew.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100