Azure / Azure/MachineLearningNotebooks

Run.complete() fails if outputs dir contains '..' in path

オープン
#1,500 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
ADO bug SDK
主要言語
Jupyter Notebook
スター
4.4k
フォーク
2.6k
PR マージ指標
30日以内にマージされた PR はありません

説明

The SDK should use `os.abspath()` ([stack overflow answer](https://stackoverflow.com/a/32838876/3842610)) before trying to upload the files.

[Reproducible example](https://github.com/swanderz/MachineLearningNotebooks/blob/19d2c7530ad6b07dcb6ff944c75135849abc4332/how-to-use-azureml/track-and-monitor-experiments/logging-api/logging-api.ipynb)

```python
from azureml.core import Experiment, Workspace, Run
import os
ws = Workspace.from_config()
experiment = Experiment(workspace=ws, name='logging-api-test')

# make a folder using the double dot to indicate parent folder
directory = '../logging-api'

if not os.path.exists(directory):
os.mkdir(directory)

# start a run
run = experiment.start_logging(outputs=directory)

# write a file to watched output folder
file_name = os.path.join(directory, "myfile.txt")
with open(file_name, "w") as f:
f.write('This is an output file that will be uploaded.\n')

# this will fail!
run2.complete()
```

stack trace
```
AzureMLAggregatedException Traceback (most recent call last)
in
----> 1 run3.complete()

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/core/run.py in complete(self, _set_status)
1341 :type _set_status: bool
1342 """
-> 1343 self._client.complete(_set_status=_set_status)
1344
1345 def fail(self, error_details=None, error_code=None, _set_status=True):

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_run_impl/run_history_facade.py in complete(self, _set_status)
592 """
593 self.flush()
--> 594 self.upload_tracked_files()
595 self.set_completed_status(_set_status=_set_status)
596

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_run_impl/run_history_facade.py in upload_tracked_files(self)
658
659 def upload_tracked_files(self):
--> 660 self.output_file_tracker.upload_tracked_files()
661
662 def take_snapshot(self, file_or_folder_path, _raise_on_validation_failure=True):

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_history/utils/context_managers.py in upload_tracked_files(self)
151
152 def upload_tracked_files(self):
--> 153 self.py_wd.track(self.artifacts_client, self.container_id, self.trackfolders, self.deny_list)
154
155

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_history/utils/context_managers.py in track(self, artifacts_client, container_id, track_folders, blacklist)
93 for fs in self.fs_list:
94 self.logger.debug("Calling track for {}".format(fs.ident()))
---> 95 fs.track(artifacts_client, container_id, track_folders, blacklist)
96 return True
97

~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_history/utils/filesystem.py in track(self, artifacts_client, container_id, track_folders, blacklist)
79
80 if len(exception_messages) > 0:
---> 81 raise AzureMLAggregatedException(exception_messages)
82
83 def _upload_folder(self, artifacts_client, container_id, path, blacklist):

AzureMLAggregatedException: AzureMLAggregatedException:
Message: Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.
StorageAccount: sasetfdevdsresvalue
ContainerName: azureml
StatusCode: 403
InnerException None
ErrorResponse
{
"error": {
"message": "Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.\n\tStorageAccount: sasetfdevdsresvalue\n\tContainerName: azureml\n\tStatusCode: 403"
}
}
```

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

Issue にリンクされている再現可能な logging-api ノートブックから始め、Run.complete() を upload_tracked_files() まで追跡します。'..' を含む outputs ディレクトリで失敗を再現します。追跡対象のファイルが正常にアップロードされ、完了時に認証エラーが報告されなくなれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
azure, jupyter-notebook, python
領域
cloud, machine-learning
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。