Azure / Azure/MachineLearningNotebooks

Run.complete() fails if outputs dir contains '..' in path

Aberta
#1,500 0 comentários 0 reações 0 responsáveis Ver no GitHub
ADO bug SDK
Linguagem predominante
Jupyter Notebook
Estrelas
4.4k
Forks
2.6k
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

The SDK should use `os.abspath()` ([stack overflow answer](https://stackoverflow.com/a/32838876/3842610)) before trying to upload the files.

[Reproducible example](https://github.com/swanderz/MachineLearningNotebooks/blob/19d2c7530ad6b07dcb6ff944c75135849abc4332/how-to-use-azureml/track-and-monitor-experiments/logging-api/logging-api.ipynb)

```python
from azureml.core import Experiment, Workspace, Run
import os
ws = Workspace.from_config()
experiment = Experiment(workspace=ws, name='logging-api-test')

# make a folder using the double dot to indicate parent folder
directory = '../logging-api'

if not os.path.exists(directory):
os.mkdir(directory)

# start a run
run = experiment.start_logging(outputs=directory)

# write a file to watched output folder
file_name = os.path.join(directory, "myfile.txt")
with open(file_name, "w") as f:
f.write('This is an output file that will be uploaded.\n')

# this will fail!
run2.complete()
```

stack trace
```
AzureMLAggregatedException Traceback (most recent call last)
in
----> 1 run3.complete()

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/core/run.py in complete(self, _set_status)
1341 :type _set_status: bool
1342 """
-> 1343 self._client.complete(_set_status=_set_status)
1344
1345 def fail(self, error_details=None, error_code=None, _set_status=True):

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_run_impl/run_history_facade.py in complete(self, _set_status)
592 """
593 self.flush()
--> 594 self.upload_tracked_files()
595 self.set_completed_status(_set_status=_set_status)
596

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_run_impl/run_history_facade.py in upload_tracked_files(self)
658
659 def upload_tracked_files(self):
--> 660 self.output_file_tracker.upload_tracked_files()
661
662 def take_snapshot(self, file_or_folder_path, _raise_on_validation_failure=True):

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_history/utils/context_managers.py in upload_tracked_files(self)
151
152 def upload_tracked_files(self):
--> 153 self.py_wd.track(self.artifacts_client, self.container_id, self.trackfolders, self.deny_list)
154
155

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_history/utils/context_managers.py in track(self, artifacts_client, container_id, track_folders, blacklist)
93 for fs in self.fs_list:
94 self.logger.debug("Calling track for {}".format(fs.ident()))
---> 95 fs.track(artifacts_client, container_id, track_folders, blacklist)
96 return True
97

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_history/utils/filesystem.py in track(self, artifacts_client, container_id, track_folders, blacklist)
79
80 if len(exception_messages) > 0:
---> 81 raise AzureMLAggregatedException(exception_messages)
82
83 def _upload_folder(self, artifacts_client, container_id, path, blacklist):

AzureMLAggregatedException: AzureMLAggregatedException:
Message: Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.
StorageAccount: sasetfdevdsresvalue
ContainerName: azureml
StatusCode: 403
InnerException None
ErrorResponse
{
"error": {
"message": "Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.\n\tStorageAccount: sasetfdevdsresvalue\n\tContainerName: azureml\n\tStatusCode: 403"
}
}
```

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Direção de pesquisa

Comece pelo notebook reproduzível de logging-api vinculado na issue e rastreie Run.complete() através de upload_tracked_files(). Reproduza a falha com um diretório outputs contendo '..'; está concluído quando o arquivo rastreado é enviado com sucesso e a conclusão não relata mais o erro de autorização.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
azure, jupyter-notebook, python
Domínio
cloud, machine-learning
Tipo de issue
Bug
Dificuldade
4/5
Tempo estimado
3-5 dias
Status de atividade
Estagnada
Clareza
Razoavelmente clara
Facilidade para iniciantes
35/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.