Azure / Azure/MachineLearningNotebooks
Use `OutputFileDatasetConfig` instead of `PipelineData` for Hyperdrive's `metrics_output`
- Lenguaje dominante
- Jupyter Notebook
- Estrellas
- 4.4k
- Forks
- 2.6k
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
`HyperDriveStep` can store the `metrics_output` into a datastore. However, this datastore location can only be specified with [`PipelineData`](https://docs.microsoft.com/en-us/python/api/azureml-pipeline-core/azureml.pipeline.core.pipelinedata?view=azure-ml-py), which is not the recommended way of moving data across pipeline steps (see [here](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-move-data-in-out-of-pipelines)).
I would love if these ways of transferring data across steps were consistent. In the meantime, how can I specify the exact location where I want the JSON file from Hyperdrive to be stored? I would like to store it within some subfolders but I don't understand how to do that. Moreover, the file created and saved by the HD step does not have an extension (it has only the name defined in the `PipelineData`) and it is stored in a generic folder `azureml` within the container:
```python
hd_data = PipelineData(
"hd_metrics",
datastore=ds,
output_mode='upload'
)
```
will save the file `hd_metrics` (no extension) in `{datastore}/azureml/{run-id}/hd_metrics`. I'd like to have more control on where and how this file is stored.
---
#### Document Details
⚠ *Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.*
* ID: 74c38a69-084f-2185-60b2-b5bdedd45c65
* Version Independent ID: d9d1234e-c713-ba08-fb19-b9874a356a5a
* Content: [azureml.pipeline.steps.HyperDriveStep class - Azure Machine Learning Python](https://docs.microsoft.com/en-us/python/api/azureml-pipeline-steps/azureml.pipeline.steps.hyperdrivestep?view=azure-ml-py)
* Content Source: [AzureML-Docset/stable/docs-ref-autogen/azureml-pipeline-steps/azureml.pipeline.steps.HyperDriveStep.yml](https://github.com/MicrosoftDocs/MachineLearning-Python-pr/blob/live/AzureML-Docset/stable/docs-ref-autogen/azureml-pipeline-steps/azureml.pipeline.steps.HyperDriveStep.yml)
* Service: **machine-learning**
* Sub-service: **core**
* GitHub Login: @DebFro
* Microsoft Alias: **debfro**
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Línea de trabajo
Comienza con AzureML-Docset/stable/docs-ref-autogen/azureml.pipeline.steps/azureml.pipeline.steps.hyperdrivestep.yml y la documentación vinculada de HyperDriveStep; después, compara las indicaciones sobre OutputFileDatasetConfig y PipelineData. Determina la configuración de metrics_output compatible y el comportamiento esperado de la ruta del datastore y del nombre de archivo; se considera terminado cuando la ubicación solicitada puede especificarse de forma coherente y queda documentada.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- machine-learning, python
- Área
- machine-learning
- Tipo de issue
- Nueva funcionalidad
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 25/100