apache / apache/druid

Coordinator documentation is stale

Open
#7,201 3 comments 0 reactions 0 assignees View on GitHub
Area - Documentation Area - Segment Balancing/Coordination
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

Coordinator documentation has the following paragraph:

> Before any unassigned segments are serviced by historical nodes, the available historical nodes for each tier are first sorted in terms of capacity, with least capacity servers having the highest priority. Unassigned segments are always assigned to the nodes with least capacity to maintain a level of balance between nodes. The coordinator does not directly communicate with a historical node when assigning it a new segment; instead the coordinator creates some temporary information about the new segment under load queue path of the historical node. Once this request is seen, the historical node will load the segment and begin servicing it.

As far as I can tell, both key pieces of information that are communicated in this paragraph are wrong:
- "Unassigned segments are always assigned to the nodes with least capacity" - no, actually regular balancing rules are in play during loading. (see `DruidCoordinatorRuleRunner`).
- "The coordinator does not directly communicate with a historical node when assigning it a new segment" - actually it does, if HTTP announcing (should we better call it "HTTP segment loading info communication"?) is used.

Could somebody please verify my conclusions?

Suggestions about how this paragraph should be rephrased are also welcome (or you can go ahead with a PR yourself).

@clintropolis @egor-ryashin @gianm

Contributor guide

Open the contributing guide

Research direction

Review the stale Coordinator paragraph and inspect DruidCoordinatorRuleRunner to verify how regular balancing rules apply during loading. Also verify the statement about direct communication when HTTP announcing is used. Done means the conclusions are confirmed and the paragraph is rephrased accurately.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.