Azure / Azure/MachineLearningNotebooks
Service deployment polling reached non-successful terminal state, current service state: Unhealthy
- 主要語言
- Jupyter Notebook
- 星號
- 4.4k
- 分支
- 2.6k
- PR 合併指標
- 30 天內沒有已合併 PR
描述
Hi,
When I mapped an SSL certificate to Azure ML endpoint with AKS compute target using the CLI commands It is successful at the time End point status is get in an Unhealthy state. we using our custom DNS and custom certificates. The cluster pods, nodes, and deployments all are in a running state and deployment was successful but the endpoint an Unhealthy state. So help to fix the issue.
Error:
Service deployment polling reached non-successful terminal state, current service state: Unhealthy
Operation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c
More information can be found using '.get_logs()'
Error:
{
"code": "DeploymentTimedOut",
"statusCode": 504,
"message": "Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087"
}
Service deployment polling reached non-successful terminal state, current service state: Unhealthy
Operation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c
More information can be found using '.get_logs()'
Error:
{
"code": "DeploymentTimedOut",
"statusCode": 504,
"message": "Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087"
}
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/azureml/core/webservice/webservice.py", line 671, in wait_for_deployment
raise WebserviceException('Service deployment polling reached non-successful terminal state, current '
azureml.exceptions._azureml_exception.WebserviceException: WebserviceException:
Message: Service deployment polling reached non-successful terminal state, current service state: Unhealthy
Operation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c
More information can be found using '.get_logs()'
Error:
{
"code": "DeploymentTimedOut",
"statusCode": 504,
"message": "Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087"
}
InnerException None
ErrorResponse
{
"error": {
"message": "Service deployment polling reached non-successful terminal state, current service state: Unhealthy\nOperation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c\nMore information can be found using '.get_logs()'\nError:\n{\n \"code\": \"DeploymentTimedOut\",\n \"statusCode\": 504,\n \"message\": \"Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087\"\n}"
}
}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3437, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 157, in
service.wait_for_deployment(show_output=True)
File "/usr/local/lib/python3.8/site-packages/azureml/core/webservice/webservice.py", line 686, in wait_for_deployment
raise WebserviceException(e.message, logger=module_logger)
azureml.exceptions._azureml_exception.WebserviceException: WebserviceException:
Message: Service deployment polling reached non-successful terminal state, current service state: Unhealthy
Operation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c
More information can be found using '.get_logs()'
Error:
{
"code": "DeploymentTimedOut",
"statusCode": 504,
"message": "Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087"
}
InnerException None
ErrorResponse
{
"error": {
"message": "Service deployment polling reached non-successful terminal state, current service state: Unhealthy\nOperation ID: 20dee957-d034-4104-8b67-8e472e6b6a6c\nMore information can be found using '.get_logs()'\nError:\n{\n \"code\": \"DeploymentTimedOut\",\n \"statusCode\": 504,\n \"message\": \"Timed out waiting for AKS deployment to complete. pollTimeout : 00:20:00 serviceName: oneforma-ocr-gpu-real-time serviceId: f94a68c5-8cf3-4532-a04d-c1adc7a48087\"\n}"
}
}
貢獻指南
這個儲存庫沒有索引到貢獻指南
研究方向
從 service.wait_for_deployment() 呼叫及其對應的 .get_logs() 輸出開始,然後檢視 AKS 部署、自訂 DNS 和自訂憑證的設定方式。當確定 Unhealthy 終端狀態的原因,並建立可重現的部署修正方案或記錄在案的限制時,該 issue 即完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- azure, kubernetes, python
- 領域
- cloud, devops, infrastructure, machine-learning
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 需要釐清
- 新手友好度
- 20/100