Azure-Samples / Azure-Samples/AI-Gateway
azure-ml-models lab - Online Deployment creation fails with InternalServerError after successful model registration
- Dominant language
- Jupyter Notebook
- Stars
- 986
- Forks
- 513
- Avg merge
- 12m
- Merged PRs (30d)
- 1
Description
@nourshaker-msft Hi Nour,
I'm running the azure-ml-models lab and I'm consistently hitting an issue in Step 4 (Register the ML model and create a deployment).
The Bicep deployment completes successfully and creates all required resources, including:
Resource Group
Azure ML Workspace
Managed Online Endpoint
APIM and Foundry resources
The ML model also registers successfully (I've tested with model versions 1, 2 and 3).
The problem occurs when creating the Azure ML Online Deployment:
az ml online-deployment create
The deployment fails with:
InternalServerError
Internal error. Please see troubleshooting guide:
https://aka.ms/oe-tsg#error-internalservererror
Troubleshooting already performed:
Verified Azure ML, Storage, Key Vault, Container Registry and related resource providers are registered.
Verified subscription quota is available in Sweden Central.
Deleted and recreated the endpoint deployments multiple times.
Tested different VM SKUs:
Standard_DS3_v2 (lab default)
Standard_B1ms
Standard_D2s_v6
Re-ran the full lab from scratch multiple times.
Confirmed the endpoint is created successfully but the deployment always ends in Failed state.
No useful deployment logs are produced and Azure ML Studio only reports an internal service error.
Have you seen this issue before or is there any recent change required in the lab for Managed Online Endpoint deployments?
Any guidance would be greatly appreciated. Thank you!
Contributor guide
Research direction
Start with the azure-ml-models lab and Step 4, then reproduce the az ml online-deployment create command after the model registration succeeds. Compare the failed deployment state with the Azure ML troubleshooting guide linked in the report and check whether the lab configuration or service behavior has changed. Done means the lab creates the managed online deployment successfully or documents a confirmed service-side cause and required update.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100