Azure / Azure/MachineLearningNotebooks

Raise exception when provisioning fails

Open
#1,524 1 comment 0 reactions 0 assignees View on GitHub
ADO bug Pipelines
Dominant language
Jupyter Notebook
Stars
4.4k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

Currently, no error is thrown when the schedule fails to provision. Only a message is printed in the console `'Provisioning failed, please try again or contact support.'`:
```python
# schedule.py
def _wait_for_provisioning(self, timeout_seconds):
"""
Wait for the completion of provisioning for this schedule.

Returns the status after the wait.

:param timeout_seconds: Number of seconds to wait before timing out.
:type timeout_seconds: int

:return: The final status.
:rtype: str
"""
status = self._schedule_provider.get_schedule_provisioning_status(self.id)
time_run = 0
sleep_period = 5
while status == 'Provisioning':
if time_run + sleep_period > timeout_seconds:
print('Timed out of waiting, status:%s.' % status, flush=True)
break
time_run += sleep_period
time.sleep(sleep_period)
status = self._schedule_provider.get_schedule_provisioning_status(self.id)
print('Provisioning status:', status)
if status == "Failed":
print('Provisioning failed, please try again or contact support.')
return status
```
If the provisioning fails, it is easy for the user to miss that. Also, there is no way the process can be safeguarded in production environments (e.g., stop the process if the schedule has not being properly provisioned). On top of this, when the provisioning fails, the schedule will still appear as `Active` (see also https://github.com/MicrosoftDocs/azure-docs/issues/77089).

---
#### Document Details

⚠ *Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.*

* ID: bdf111d3-ff02-238f-68d9-5a36bb3bf75b
* Version Independent ID: da555131-26d9-c768-d8d4-e46c485d2601
* Content: [azureml.pipeline.core.schedule.Schedule class - Azure Machine Learning Python](https://docs.microsoft.com/en-us/python/api/azureml-pipeline-core/azureml.pipeline.core.schedule.schedule?view=azure-ml-py)
* Content Source: [AzureML-Docset/stable/docs-ref-autogen/azureml-pipeline-core/azureml.pipeline.core.schedule.Schedule.yml](https://github.com/MicrosoftDocs/MachineLearning-Python-pr/blob/live/AzureML-Docset/stable/docs-ref-autogen/azureml-pipeline-core/azureml.pipeline.core.schedule.Schedule.yml)
* Service: **machine-learning**
* Sub-service: **core**
* GitHub Login: @DebFro
* Microsoft Alias: **debfro**

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.