Azure / Azure/MachineLearningNotebooks

Creating a file dataset from a single directory in datastore requires azureml-dataset-runtime?

Ouverte
#1,912 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Jupyter Notebook
Étoiles
4.4k
Forks
2.6k
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

I am trying to create a file dataset from a single directory in datastore. Im following the code block from

https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.data.dataset_factory.filedatasetfactory?view=azure-ml-py#azureml-data-dataset-factory-filedatasetfactory-from-files

Specifically,

```python
from azureml.core import Dataset, Datastore

# create file dataset from a single file in datastore
datastore = Datastore.get(workspace, 'workspaceblobstore')

# create file dataset from a single directory in datastore
file_dataset_2 = Dataset.File.from_files(path=(datastore, 'image/'))
```

However, when I try to replicate these steps for my own Datastore, I encounter an Import Error

`ImportError: Missing required package "azureml-dataset-runtime", which can be installed by running: "c:\Users\\.conda\envs\\python.exe" -m pip install azureml-dataset-runtime --upgrade`

I am on Python 3.11.3 and I tried installing azureml-dataset-runtime but I encounter a dependency clash which requires me to downgrade to Python 3.8.

Furthermore, from the PyPI page

https://pypi.org/project/azureml-dataset-runtime/

It states that azureml-dataset-runtime is "is internal, and is not intended to be used directly."

Is this intended? I am trying to mount my data for a custom ML training job, using the [as_mount](https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.data.filedataset?view=azure-ml-py#azureml-data-filedataset-as-mount) function from the FileDataset Class. Please let me know if there is a better alternative to mounting data, or am I forced to use Python 3.8?

---
#### Document Details

⚠ *Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.*

* ID: 091afd7e-72ca-a384-01db-4da4d40a6734
* Version Independent ID: 0f3783bf-ab1f-f0d6-08f3-90becae914e8
* Content: [azureml.data.dataset_factory.FileDatasetFactory class - Azure Machine Learning Python](https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.data.dataset_factory.filedatasetfactory?view=azure-ml-py#azureml-data-dataset-factory-filedatasetfactory-from-files)
* Content Source: [AzureML-Docset/stable/docs-ref-autogen/azureml-core/azureml.data.dataset_factory.FileDatasetFactory.yml](https://github.com/MicrosoftDocs/MachineLearning-Python-pr/blob/live/AzureML-Docset/stable/docs-ref-autogen/azureml-core/azureml.data.dataset_factory.FileDatasetFactory.yml)
* Service: **machine-learning**
* Sub-service: **core**
* GitHub Login: @DebFro
* Microsoft Alias: **debfro**

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Piste de recherche

Commencez par l’exemple de documentation de FileDatasetFactory.from_files et le point d’entrée FileDataset.as_mount, puis vérifiez comment l’exemple se comporte avec Python 3.11 et la dépendance azureml-dataset-runtime. Le travail est terminé lorsque le workflow documenté pour un répertoire unique dispose d’un runtime et d’un chemin de dépendances pris en charge, ou lorsque la documentation explique clairement la limitation et une approche alternative de montage.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
python
Domaine
documentation, machine-learning
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
À l'abandon
Clarté
Plutôt claire
Accessibilité débutants
25/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.