Coordinator documentation is stale
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
Coordinator documentation has the following paragraph:
> Before any unassigned segments are serviced by historical nodes, the available historical nodes for each tier are first sorted in terms of capacity, with least capacity servers having the highest priority. Unassigned segments are always assigned to the nodes with least capacity to maintain a level of balance between nodes. The coordinator does not directly communicate with a historical node when assigning it a new segment; instead the coordinator creates some temporary information about the new segment under load queue path of the historical node. Once this request is seen, the historical node will load the segment and begin servicing it.
As far as I can tell, both key pieces of information that are communicated in this paragraph are wrong:
- "Unassigned segments are always assigned to the nodes with least capacity" - no, actually regular balancing rules are in play during loading. (see `DruidCoordinatorRuleRunner`).
- "The coordinator does not directly communicate with a historical node when assigning it a new segment" - actually it does, if HTTP announcing (should we better call it "HTTP segment loading info communication"?) is used.
Could somebody please verify my conclusions?
Suggestions about how this paragraph should be rephrased are also welcome (or you can go ahead with a PR yourself).
@clintropolis @egor-ryashin @gianm
Contributor guide
Research direction
Review the stale Coordinator paragraph and inspect DruidCoordinatorRuleRunner to verify how regular balancing rules apply during loading. Also verify the statement about direct communication when HTTP announcing is used. Done means the conclusions are confirmed and the paragraph is rephrased accurately.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100