Azure / Azure/MachineLearningNotebooks

TrainingOutput with OutputFileDatasetConfig (how to retrieve best HyperDriveStep run)

Open
#1,532 1 comment 0 reactions 0 assignees View on GitHub
ADO bug MLOps
Dominant language
Jupyter Notebook
Stars
4.4k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

I am using HyperDrive in my pipeline via the `HyperDriveStep`. How can I export the model from the best HyperDrive run, register it, and use it in a following step?

In the doc, you refer to `TrainingOutput`, which it seems can be used only with `PipelineData`. In my pipeline, instead, I am using `OutputFileDatasetConfig` to move data across steps, which is also the recommended way https://github.com/MicrosoftDocs/azure-docs/issues/76169.

Currently, the _training step_ is storing the pickled model in a blob storage after each child run is completed. In the blob storage, each model is stored in a folder named after the {run_id}, for example `HD_7bf73de6-90dd-44dc-a8a4-8e8fd3f481f6_xxx`, where `xxx` is the HyperDrive iteration (0, 1, 2,...). In a subsequent pipeline step, I would like to retrieve the best model but I need to know _HyperDriver step_ best run ID.

In a step dedicated to register/use the best model, I tried
```python
# step to retrieve the best Hyperdrive model
run = Run.get_context()
pipeline_run = run.parent
hyperdrive_step = pipeline_run.find_step_run("hyperdrive_step")[0]
best_run = hyperdrive_step.get_best_run_by_primary_metric()
```
this fails, as `hyperdrive_step` is a `StepRun` object and it does not have the method `get_best_run_by_primary_metric`.

Would you be able to help?

---
#### Document Details

⚠ *Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.*

* ID: 6d30ceef-a59a-3b5e-3ae0-e60371f47f88
* Version Independent ID: 28652f23-4cde-43ea-376f-44e8af33dff3
* Content: [azureml.pipeline.core.TrainingOutput class - Azure Machine Learning Python](https://docs.microsoft.com/en-us/python/api/azureml-pipeline-core/azureml.pipeline.core.trainingoutput?view=azure-ml-py)
* Content Source: [AzureML-Docset/stable/docs-ref-autogen/azureml-pipeline-core/azureml.pipeline.core.TrainingOutput.yml](https://github.com/MicrosoftDocs/MachineLearning-Python-pr/blob/live/AzureML-Docset/stable/docs-ref-autogen/azureml-pipeline-core/azureml.pipeline.core.TrainingOutput.yml)
* Service: **machine-learning**
* Sub-service: **core**
* GitHub Login: @DebFro
* Microsoft Alias: **debfro**

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.