Azure / Azure/MachineLearningNotebooks

Creating a file dataset from a single directory in datastore requires azureml-dataset-runtime?

未關閉
#1,912 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

主要語言
Jupyter Notebook
星號
4.4k
分支
2.6k
PR 合併指標
30 天內沒有已合併 PR

描述

I am trying to create a file dataset from a single directory in datastore. Im following the code block from

https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.data.dataset_factory.filedatasetfactory?view=azure-ml-py#azureml-data-dataset-factory-filedatasetfactory-from-files

Specifically,

from azureml.core import Dataset, Datastore

 # create file dataset from a single file in datastore
 datastore = Datastore.get(workspace, 'workspaceblobstore')

 # create file dataset from a single directory in datastore
 file_dataset_2 = Dataset.File.from_files(path=(datastore, 'image/'))

However, when I try to replicate these steps for my own Datastore, I encounter an Import Error

ImportError: Missing required package "azureml-dataset-runtime", which can be installed by running: "c:\Users\<user>\.conda\envs\<my-conda-env-name>\python.exe" -m pip install azureml-dataset-runtime --upgrade

I am on Python 3.11.3 and I tried installing azureml-dataset-runtime but I encounter a dependency clash which requires me to downgrade to Python 3.8.

Furthermore, from the PyPI page

https://pypi.org/project/azureml-dataset-runtime/

It states that azureml-dataset-runtime is "is internal, and is not intended to be used directly."

Is this intended? I am trying to mount my data for a custom ML training job, using the as_mount function from the FileDataset Class. Please let me know if there is a better alternative to mounting data, or am I forced to use Python 3.8?


Document Details

Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

貢獻指南

這個儲存庫沒有索引到貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

從 FileDatasetFactory.from_files 的文件範例和 FileDataset.as_mount 進入點開始,接著檢查該範例在 Python 3.11 和 azureml-dataset-runtime 相依性下的行為。完成的標準是:文件所述的單一目錄工作流程具有受支援的執行階段和相依性路徑,或文件清楚說明限制以及替代的掛載方法。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
documentation, machine-learning
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
25/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。