Azure / Azure/azure-cli-extensions
ModuleNotFoundError when deploying Azure ML model to online endpoint
- Dominant language
- Python
- Stars
- 454
- Forks
- 1.7k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 64
Description
### Describe the bug
I am trying to deploy my Azure ML model to an online endpoint hosted on Azure, as described in the documentation https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-online-endpoints?view=azureml-api-2&tabs=azure-cli. However, I'm failing to import one of my files that should be included in the code repository where the scoring script is located. The logic for inferencing is quite complicated, more so than in the above linked documentation, so additional files must be included apart from just the scoring script.
### Related command
`az ml online-deployment create -f azure/deployment.yml --resource-group --workspace-name `
I have deployed the exact same scoring script, and everything else the same, locally and it works perfectly fine. The issue is only when trying to deploy to my Azure endpoint.
### Errors
The deployment fails with ResourceNotReady. Upon inspection of the logs in Azure Studio, the error is:
2023-09-29 19:00:24,282 E [65] azmlinfsrv - Traceback (most recent call last):
File "/azureml-envs/azureml_de0eccae1d7809be16f4cee8f4e60c52/lib/python3.10/site-packages/azureml_inference_server_http/server/user_script.py", line 77, in load_script
main_module_spec.loader.exec_module(user_module)
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/var/azureml-app/rebel-core-deployment/scoring_script.py", line 8, in
from utils.post_processing import post_process
ModuleNotFoundError: No module named 'utils.post_processing'; 'utils' is not a package
The directory structure is as follows:
root:
- \_\_init\_\_.py
- scoring_script.py
- utils/
- \_\_init\_\_.py
- post_processing.py
- models/
- \_\_init\_\_.py
- StrengthNet.py
- RateNet.py
- azure/
- deployment.yml
### Issue script & Debug output
My scoring script is as follows:
```
import json
import pandas as pd
import torch
from models.RateNet import RateNet
from models.StrengthNet import StrengthNet
from utils.post_processing import post_process
strength_net = None
rate_net = None
def init():
global config
global strength_net
global rate_net
# Initialization
# Omitted for business sensitivity
def predict(data, metadata):
# Omitted ...
return predictions
def run(request):
request = json.loads(request)
data = torch.tensor(request["data"])
metadata = torch.tensor(request["metadata"])
# Omitted
predictions = predict(data, metadata)
return predictions.tolist()
```
Note that the first two imports work, which are also local files. The program only crashes on the utils package.
Deployment YAML file:
```
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: deployment-name-omitted
endpoint_name: endpoint-name-omitted
model: azureml:ModelNameOmitted:2
code_configuration:
code: ../
scoring_script: scoring_script.py
environment: azureml:env-name-omitted:14
instance_type: Standard_DS4_v2
instance_count: 1
```
### Expected behavior
The expected behaviour is that the deployment is successful, and that the utils.post_processing file is successfully imported. When deployed locally, with the --local tag, things work fine. Additionally, the deployment successfully imports from files in the models directory, so there should logically not be any difference to the utils directory.
### Environment Summary
azure-cli 2
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.