Overlord can support many autoscalers of different categories
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Motivation
Since #7066 Druid supports different categories of tasks which run on dedicated instance of MiddleManager. But Overlord still can spawn only one instance of MiddleManager via Autoscaler. That is not flexible when Druid should support different categories of tasks and cluster parameter for the each of the category is different. For instance realtime and batch tasks will require clusters of different configurations: workers count; CPU, memory, per each worker, etc. In that case overlord could provision dedicated cluster per each task category.
### Proposed changes
Create `CategoriedWorkerBehaviorConfig` to support declaration of several autoscalers. Each of the autoscalers should have "category" property which should link it to according "category" of `Task` based on `CategoryConfig`.
Change all current implementations of `ProvisioningStrategy` interface to select appropriate `AutoScaler` from `CategoriedWorkerBehaviorConfig` during "provisioning" or "terminating". In that way overlord can spawn/terminate a corresponding cluster for a submitted task. If there is no matched autoscaler found in the config then the default one will be used. If there is no declaration of default autoscaler then there will be no autoscaling and tasks will be in `pending` state.
### Rationale
Structure of the `CategoriedWorkerBehaviorConfig` will be like this:
```
{
"selectStrategy": {
"type": "fillCapacity",
...
},
"autoScalers": [
{
"type": "ec2",
"minNumWorkers": 0,
"maxNumWorkers: 5,
"envConfig": {
"availabilityZone": "us-east-1a"
...
}
...
},
{
"type": "ec2",
"category": "_category_1_",
"minNumWorkers": 2,
"maxNumWorkers: 10,
"envConfig": {
"availabilityZone": "us-west-2a"
...
}
...
},
{
"type": "ec2",
"category": "_category_1_",
"minNumWorkers": 1,
"maxNumWorkers: 3,
"envConfig": {
"availabilityZone": "us-east-2a"
...
}
...
}
]
}
```
The "autoScalers" section will contain definitions of all supported autoscalers as array.
Where "_category_1_" and "_category_2_" are category names according to supported types of tasks (and middleManagers). Default autoscaler will not have category name and its declaration is optional. If it was declared then any task which doesn't have autoscaler for its category will be served by default autoscaler.
### Operational impact
No impact because this PR will not change the existing config files.
Contributor guide
Research direction
Start with the ProvisioningStrategy interface and its current implementations, then inspect CategoryConfig and the existing worker behavior configuration. Done means provisioning and termination select a matching category autoscaler or the default, while tasks remain pending when neither is configured.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100