Azure / Azure/azure-cli-extensions

'az ml computetarget create aks' should support v2 and v3 GPU VM families

Open
#512 4 comments 0 reactions 0 assignees View on GitHub
extension/ml Service Attention
Dominant language
Python
Stars
454
Forks
1.7k
Avg merge
2d 19h
Merged PRs (30d)
64

Description

**Is your feature request related to a problem? Please describe.**
I'm unable to create an AKS cluster with the `az ml computetarget create aks` command using node types from v2 or v3 GPU families, such as `Standard_NC6s_v3`.
According to [AKS ARM provider reference](https://docs.microsoft.com/en-us/azure/templates/microsoft.containerservice/2018-03-31/managedclusters#managedclusteragentpoolprofile-object), AKS supports NC v2 and v3 machines. Yet, when I try to create the cluster using the Azure ML extension, I receive a parameter validation error claiming that only v1 NC machines are supported:
```
> az ml computetarget create aks --name myservice-v100 --agent-count 2 --vm-size Standard_NC6s_v3
Provisioning compute resources...
Resource creation submitted successfully.
Provisioning operation finished, operation "Failed"
Async operation failed with
StatusCode: 400
Message: The request is invalid
Compute object has provisioning state "Failed"
and provisioning errors: [{'code': 'BadRequest', 'message': 'The request is invalid', 'error': {'code': 'BadRequest', 'statusCode': 400, 'message': 'The request is invalid', 'details': [{'code': 'The request is invalid', 'message': "RequestId: 7293df66-0e58-4df7-a0d9-9dc96c544673\nError code: 'InvalidTemplate'. Target: ''. Message: 'Deployment template validation failed: 'The provided value 'Standard_NC6s_v3' for the template parameter 'agentVMSize' at line '1' and column '972' is not valid. The parameter value is not part of the allowed value(s): 'Standard_A10,Standard_A11,Standard_A2,Standard_A2_v2,Standard_A2m_v2,Standard_A3,Standard_A4,Standard_A4_v2,Standard_A4m_v2,Standard_A5,Standard_A6,Standard_A7,Standard_A8,Standard_A8_v2,Standard_A8m_v2,Standard_A9,Standard_D11,Standard_D11_v2,Standard_D12,Standard_D12_v2,Standard_D13,Standard_D13_v2,Standard_D14,Standard_D14_v2,Standard_D15_v2,Standard_D2,Standard_D2_v2,Standard_D3,Standard_D3_v2,Standard_D4,Standard_D4_v2,Standard_D5_v2,Standard_DS11,Standard_DS11_v2,Standard_DS12,Standard_DS12_v2,Standard_DS13,Standard_DS13_v2,Standard_DS14,Standard_DS14_v2,Standard_DS15_v2,Standard_DS2,Standard_DS2_v2,Standard_DS3,Standard_DS3_v2,Standard_DS4,Standard_DS4_v2,Standard_DS5_v2,Standard_F16,Standard_F16s,Standard_F2,Standard_F2s,Standard_F4,Standard_F4s,Standard_F8,Standard_F8s,Standard_G1,Standard_G2,Standard_NC6,Standard_NC12,Standard_NC24,Standard_NC24r,Standard_NV6,Standard_NV12,Standard_NV24,Standard_B1s,Standard_B1ms,Standard_B2s,Standard_B2ms,Standard_B4ms,Standard_B8ms'.'.'\n"}]}}]
```

**Describe the solution you'd like**
`az ml computetarget create aks` should support all GPU node types which are supported by AKS.

**Describe alternatives you've considered**
I can probably create the AKS cluster myself and attach it to the ML Workspace using `az ml computetarget attach aks`, but it will be much more complicated than simply creating the compute target with one command.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.