Azure-Samples / Azure-Samples/AI-Gateway

azure-ml-models lab - Online Deployment creation fails with InternalServerError after successful model registration

Open
#393 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
986
Forks
513
Avg merge
12m
Merged PRs (30d)
1

Description

@nourshaker-msft Hi Nour,

I'm running the azure-ml-models lab and I'm consistently hitting an issue in Step 4 (Register the ML model and create a deployment).

The Bicep deployment completes successfully and creates all required resources, including:

Resource Group
Azure ML Workspace
Managed Online Endpoint
APIM and Foundry resources

The ML model also registers successfully (I've tested with model versions 1, 2 and 3).

The problem occurs when creating the Azure ML Online Deployment:

az ml online-deployment create

The deployment fails with:

InternalServerError
Internal error. Please see troubleshooting guide:
https://aka.ms/oe-tsg#error-internalservererror

Troubleshooting already performed:

Verified Azure ML, Storage, Key Vault, Container Registry and related resource providers are registered.
Verified subscription quota is available in Sweden Central.
Deleted and recreated the endpoint deployments multiple times.
Tested different VM SKUs:
Standard_DS3_v2 (lab default)
Standard_B1ms
Standard_D2s_v6
Re-ran the full lab from scratch multiple times.
Confirmed the endpoint is created successfully but the deployment always ends in Failed state.

No useful deployment logs are produced and Azure ML Studio only reports an internal service error.

Have you seen this issue before or is there any recent change required in the lab for Managed Online Endpoint deployments?

Any guidance would be greatly appreciated. Thank you!

Contributor guide

Open the contributing guide

Research direction

Start with the azure-ml-models lab and Step 4, then reproduce the az ml online-deployment create command after the model registration succeeds. Compare the failed deployment state with the Azure ML troubleshooting guide linked in the report and check whether the lab configuration or service behavior has changed. Done means the lab creates the managed online deployment successfully or documents a confirmed service-side cause and required update.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.