dask / dask/dask-cloudprovider
Add create cluster support for ECSCluster on EC2
- Dominant language
- Python
- Stars
- 147
- Forks
- 119
- PR merge metrics
- No merged PRs in 30d
Description
Wanted to open this up for discussion. How much should ECSCluster take care of for the user on the topic of creating a cluster.
There's generally two ways of going about doing this:
- ECSCluster takes in a cluster that already exists and that is the only way it works. It does _not_ manage a cluster for you. This is how it is implemented currently.
- ECSCluster, if not given an existing cluster ARN, will create an autoscaling group of EC2's and attach it to the ECS cluster that it creates. It would use user provided specifications or create the cluster based on reasonable defaults. This is similar to how FargateCluster works.
For the former, nothing needs to be done as ECSCluster already handles this. The latter requires discussion on how opinionated ECSCluster should be with _how_ it creates the cluster since there are a couple different ways of spinning up resources in AWS (boto, terraform, cloudformation, ... to name a few)
Ideas that have been presented already:
- We don't do this at all (earlier in this issue)
- A cluster bootstrap stage is handled by a separate library or class?
- Basic cluster bootstrap step is included in the library (perhaps with boto?)
Contributor guide
Assessment
This issue has not been assessed yet.