dask / dask/dask-cloudprovider

Add support for spot termination notices

Open
#48 1 comment 2 reactions 0 assignees View on GitHub
enhancement provider/aws/ecs
Dominant language
Python
Stars
147
Forks
119
PR merge metrics
No merged PRs in 30d

Description

Following on from discussions in dask/distributed#3300 it would be great if workers gracefull closed when they recieve a spot termination notification. This will be even more useful with #47.

- `ECSCluster` and `FargateCluster` workers should start with a `--preload 'dask_cloudprovider.aws.spot_termination_handler'` or similar. This could also be used for manual cluster creation separately from close cluster managers.
- This new `dask_cloudprovider.aws.spot_termination_handler` module should attach a task to the workers `periodic_callbacks` to poll the Spot Termination endpoint.
- When it gets a spot termination notice it should call `close_gracefully` on the worker.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.