iterative / iterative/cml

`cml runner`: Request spot instances from requirements

Open
#1,101 5 comments 3 reactions 0 assignees View on GitHub
cml-runner enhancement external-request
Dominant language
JavaScript
Stars
4.2k
Forks
345
PR merge metrics
No merged PRs in 30d

Description

## What?

Would it be possible to add the ability to request spot instances from a list of requirements rather than an instance type or a GPU type?

For example, I would like to tell cml runner, I want an instance at the lowest price that:
- has 2 nvidia GPUs
- has at least 8 GB of ram
- is the latest instance generation
- is in any availability zone
- etc.

(more context: discord#cml/1000042237830373406)

## Why?

Spot instances are not available 100% of the time and as explained in the [aws best practices guide](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-best-practices.html#be-instance-type-flexible), the less constraints, the more chance we have to fulfil our spot instance request.

## Possible solutions

I think we have multiple way of implementing it.

The first and low cost solution would be to allow multiple value for the `--cloud-type` option:

```bash
cml runner
--cloud-spot
--cloud-type=g3.4xlarge,g4dn.xlarge,g5.8xlarge
```

The requirements to instance type conversion would need to be done beforehand. But after all, instance types don't change often.

-----

The second solution would be to implement all the requirement logic into cml runner. Not sure what the api could look like but something like this could be useful:

```bash
cml runner
--cloud-spot
--cloud-spot-requirement="AcceleratorCount>=1"
--cloud-spot-requirement="AcceleratorManufacturers=NVIDIA"
...
```

-----

Third solution (basically the second one but probably easier to implement):

```json
{
"AcceleratorCount": {
"Min": 1
},
"AcceleratorManufacturers": [
"nvidia"
]
}
```

```bash
cml runner
--cloud-spot
--cloud-spot-json-requirements=path_to_requirements.json
...
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.