iterative / iterative/cml

Cannot create certain GCP GPU instances

Open
#1,398 11 comments 0 reactions 1 assignee Claimed by @0x2b3bfa0 View on GitHub
bug cloud-gcp
Dominant language
JavaScript
Stars
4.2k
Forks
345
PR merge metrics
No merged PRs in 30d

Description

Hi everyone,

I'm getting the following error when trying to create A100 or L4 based instances on GCP using `cml runner launch` (`a2-highgpu` and `g2-standard` types respectively):

```
***"level":"error","message":"terraform error: Error: Failed creating the machine: googleapi: Error 400: Instances with guest accelerators do not support live migration., badRequest"***
```

I have no problem creating V100 and T4 instances (both `n1` types).

I have found [this](https://groups.google.com/g/gce-discussion/c/e9K3h3fQuJk/m/UxyKqskLAQAJ) discussion, which suggests the maintenance policy needs to be set to `TERMINATE`. Am I on the right track, and if yes, is there a way to do that using `cml runner launch`?

Regards,
Alex.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.