Azure / Azure/azureml-examples
endpoints/online/managed: how to deploy with multiple models / subfolders
- Dominant language
- Jupyter Notebook
- Stars
- 2k
- Forks
- 1.7k
- Avg merge
- 18h 18m
- Merged PRs (30d)
- 2
Description
## Which example? Describe the issue
We are trying to test migration from SDKv1 deployment to v2.
In the SDKv1 scenario we would use the SDK to download models to a local `azureml-models`, which would then be included in the model definition. Ex below:
```
./local/azureml-models
/model-name-1
/1
some-somenet.onnx
/2
somenet.onnx
/model-another-b
/3
another.onnx
```
This folder structure is the one created by azureml sdk when downloading models in preparation for a deployment.
```yaml
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: ep-name
model:
path: ../../azureml-models
...
```
When trying to reproduce this with online endpoints, using for instance, [cli/endpoints/online/managed](https://github.com/Azure/azureml-examples/blob/main/cli/deploy-safe-rollout-online-endpoints.sh), we receive the following message:
`Directory /.../.../azureml-models is empty. path or local_path must be a non-empty directory.`
Going through the documentation here https://docs.microsoft.com/en-us/azure/machine-learning/how-to-deploy-managed-online-endpoints#use-more-than-one-model there is a mention to creating a model explicitly pointing to a folder with multiple models.
However, it seems that `az ml model create -f model-bundle.yml` produces a similar error
```yaml
$schema: https://azuremlschemas.azureedge.net/latest/model.schema.json
name: model-bundle-1
path: ../../azureml-models
```
```bash
$ az ml model create -f model-bundle.yml --resource-group my_rg --subscription my_sub --workspace-name my_ws
Directory /.../.../azureml-models is empty. path or local_path must be a non-empty directory.
```
Any idea how CLIv2 can be used to address this scenario?
Thanks
Contributor guide
Research direction
Start with cli/endpoints/online/managed/deploy-safe-rollout-online-endpoints.sh and the linked managed online endpoints documentation, then reproduce the model-bundle.yml command against the described azureml-models structure. Done should clarify or demonstrate how CLIv2 supports deploying multiple models in subfolders, including a working example or an explicit limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- cli, cloud, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100