Make SpecCluster declarative
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
`SpecCluster` and other components within distributed like `distributed.cli.dask_spec` aim to create clusters and workers according to a specification.
Currently these operations are performed imperatively. The `SpecCluster` class is given a spec and a resource is created to match the spec.
I would like to move things towards a declarative approach instead. This would mean the class is given a spec and it ensures that a resource exists to match that spec. If one doesn't exist it will create it, same as today. But if it already exists it will do nothing.
This slight difference of **ensuring** a resource instead of **creating** one will enable much more flexibility when it comes to recreating objects to represent existing resources. This ties into the lifecycle work I've been doing as part of #4263.
Opening this issue to signal my intent to work on this.
Contributor guide
Assessment
This issue has not been assessed yet.