Azure / Azure/MachineLearningNotebooks

Service deployment polling reached non-successful terminal state, current service state: Unhealthy

Ouverte
#1,767 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Jupyter Notebook
Étoiles
4.4k
Forks
2.6k
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

Hi,

When I mapped an SSL certificate to Azure ML endpoint with AKS compute target using the CLI commands It is successful at the time End point status is get in an Unhealthy state. we using our custom DNS and custom certificates. The cluster pods, nodes, and deployments all are in a running state and deployment was successful but the endpoint an Unhealthy state. So help to fix the issue.

Error:
Service deployment polling reached non-successful terminal state, current service state: Unhealthy
Operation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c
More information can be found using '.get_logs()'
Error:
{
"code": "DeploymentTimedOut",
"statusCode": 504,
"message": "Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087"
}
Service deployment polling reached non-successful terminal state, current service state: Unhealthy
Operation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c
More information can be found using '.get_logs()'
Error:
{
"code": "DeploymentTimedOut",
"statusCode": 504,
"message": "Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087"
}
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/azureml/core/webservice/webservice.py", line 671, in wait_for_deployment
raise WebserviceException('Service deployment polling reached non-successful terminal state, current '
azureml.exceptions._azureml_exception.WebserviceException: WebserviceException:
Message: Service deployment polling reached non-successful terminal state, current service state: Unhealthy
Operation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c
More information can be found using '.get_logs()'
Error:
{
"code": "DeploymentTimedOut",
"statusCode": 504,
"message": "Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087"
}
InnerException None
ErrorResponse
{
"error": {
"message": "Service deployment polling reached non-successful terminal state, current service state: Unhealthy\nOperation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c\nMore information can be found using '.get_logs()'\nError:\n{\n \"code\": \"DeploymentTimedOut\",\n \"statusCode\": 504,\n \"message\": \"Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087\"\n}"
}
}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3437, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 157, in
service.wait_for_deployment(show_output=True)
File "/usr/local/lib/python3.8/site-packages/azureml/core/webservice/webservice.py", line 686, in wait_for_deployment
raise WebserviceException(e.message, logger=module_logger)
azureml.exceptions._azureml_exception.WebserviceException: WebserviceException:
Message: Service deployment polling reached non-successful terminal state, current service state: Unhealthy
Operation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c
More information can be found using '.get_logs()'
Error:
{
"code": "DeploymentTimedOut",
"statusCode": 504,
"message": "Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087"
}
InnerException None
ErrorResponse
{
"error": {
"message": "Service deployment polling reached non-successful terminal state, current service state: Unhealthy\nOperation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c\nMore information can be found using '.get_logs()'\nError:\n{\n \"code\": \"DeploymentTimedOut\",\n \"statusCode\": 504,\n \"message\": \"Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087\"\n}"
}
}

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Piste de recherche

Commencez par l’appel service.wait_for_deployment() et la sortie .get_logs() correspondante, puis examinez la configuration du déploiement AKS, du DNS personnalisé et du certificat personnalisé. L’issue est terminée lorsque la cause de l’état terminal Unhealthy est identifiée et qu’un correctif de déploiement reproductible ou une limitation documentée est établi.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
azure, kubernetes, python
Domaine
cloud, devops, infrastructure, machine-learning
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
À l'abandon
Clarté
À clarifier
Accessibilité débutants
20/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.