Azure / Azure/MachineLearningNotebooks
Use `OutputFileDatasetConfig` instead of `PipelineData` for Hyperdrive's `metrics_output`
- 主要語言
- Jupyter Notebook
- 星號
- 4.4k
- 分支
- 2.6k
- PR 合併指標
- 30 天內沒有已合併 PR
描述
`HyperDriveStep` can store the `metrics_output` into a datastore. However, this datastore location can only be specified with [`PipelineData`](https://docs.microsoft.com/en-us/python/api/azureml-pipeline-core/azureml.pipeline.core.pipelinedata?view=azure-ml-py), which is not the recommended way of moving data across pipeline steps (see [here](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-move-data-in-out-of-pipelines)).
I would love if these ways of transferring data across steps were consistent. In the meantime, how can I specify the exact location where I want the JSON file from Hyperdrive to be stored? I would like to store it within some subfolders but I don't understand how to do that. Moreover, the file created and saved by the HD step does not have an extension (it has only the name defined in the `PipelineData`) and it is stored in a generic folder `azureml` within the container:
```python
hd_data = PipelineData(
"hd_metrics",
datastore=ds,
output_mode='upload'
)
```
will save the file `hd_metrics` (no extension) in `{datastore}/azureml/{run-id}/hd_metrics`. I'd like to have more control on where and how this file is stored.
---
#### Document Details
⚠ *Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.*
* ID: 74c38a69-084f-2185-60b2-b5bdedd45c65
* Version Independent ID: d9d1234e-c713-ba08-fb19-b9874a356a5a
* Content: [azureml.pipeline.steps.HyperDriveStep class - Azure Machine Learning Python](https://docs.microsoft.com/en-us/python/api/azureml-pipeline-steps/azureml.pipeline.steps.hyperdrivestep?view=azure-ml-py)
* Content Source: [AzureML-Docset/stable/docs-ref-autogen/azureml-pipeline-steps/azureml.pipeline.steps.HyperDriveStep.yml](https://github.com/MicrosoftDocs/MachineLearning-Python-pr/blob/live/AzureML-Docset/stable/docs-ref-autogen/azureml-pipeline-steps/azureml.pipeline.steps.HyperDriveStep.yml)
* Service: **machine-learning**
* Sub-service: **core**
* GitHub Login: @DebFro
* Microsoft Alias: **debfro**
貢獻指南
這個儲存庫沒有索引到貢獻指南
研究方向
從 AzureML-Docset/stable/docs-ref-autogen/azureml.pipeline.steps/azureml.pipeline.steps.hyperdrivestep.yml 和連結的 HyperDriveStep 文件開始,接著比較 OutputFileDatasetConfig 與 PipelineData 的指南。確定受支援的 metrics_output 設定,以及預期的 datastore 路徑與檔名行為;當要求的位置可以一致地指定並完成文件化時,即表示完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- machine-learning, python
- 領域
- machine-learning
- Issue 類型
- 功能
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 基本清楚
- 新手友好度
- 25/100