Azure / Azure/azureml-examples

endpoints/online/managed: how to deploy with multiple models / subfolders

Open
#1,328 0 comments 0 reactions 0 assignees View on GitHub
example issue
Dominant language
Jupyter Notebook
Stars
2k
Forks
1.7k
Avg merge
18h 18m
Merged PRs (30d)
2

Description

## Which example? Describe the issue
We are trying to test migration from SDKv1 deployment to v2.

In the SDKv1 scenario we would use the SDK to download models to a local `azureml-models`, which would then be included in the model definition. Ex below:
```
./local/azureml-models
/model-name-1
/1
some-somenet.onnx
/2
somenet.onnx
/model-another-b
/3
another.onnx
```
This folder structure is the one created by azureml sdk when downloading models in preparation for a deployment.

```yaml
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: ep-name
model:
path: ../../azureml-models
...
```

When trying to reproduce this with online endpoints, using for instance, [cli/endpoints/online/managed](https://github.com/Azure/azureml-examples/blob/main/cli/deploy-safe-rollout-online-endpoints.sh), we receive the following message:
`Directory /.../.../azureml-models is empty. path or local_path must be a non-empty directory.`

Going through the documentation here https://docs.microsoft.com/en-us/azure/machine-learning/how-to-deploy-managed-online-endpoints#use-more-than-one-model there is a mention to creating a model explicitly pointing to a folder with multiple models.

However, it seems that `az ml model create -f model-bundle.yml` produces a similar error
```yaml
$schema: https://azuremlschemas.azureedge.net/latest/model.schema.json
name: model-bundle-1
path: ../../azureml-models
```
```bash
$ az ml model create -f model-bundle.yml --resource-group my_rg --subscription my_sub --workspace-name my_ws
Directory /.../.../azureml-models is empty. path or local_path must be a non-empty directory.
```

Any idea how CLIv2 can be used to address this scenario?
Thanks

Contributor guide

Open the contributing guide

Research direction

Start with cli/endpoints/online/managed/deploy-safe-rollout-online-endpoints.sh and the linked managed online endpoints documentation, then reproduce the model-bundle.yml command against the described azureml-models structure. Done should clarify or demonstrate how CLIv2 supports deploying multiple models in subfolders, including a working example or an explicit limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
cli, cloud, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.