Azure / Azure/MachineLearningNotebooks
Run.complete() fails if outputs dir contains '..' in path
- 主要語言
- Jupyter Notebook
- 星號
- 4.4k
- 分支
- 2.6k
- PR 合併指標
- 30 天內沒有已合併 PR
描述
The SDK should use `os.abspath()` ([stack overflow answer](https://stackoverflow.com/a/32838876/3842610)) before trying to upload the files.
[Reproducible example](https://github.com/swanderz/MachineLearningNotebooks/blob/19d2c7530ad6b07dcb6ff944c75135849abc4332/how-to-use-azureml/track-and-monitor-experiments/logging-api/logging-api.ipynb)
```python
from azureml.core import Experiment, Workspace, Run
import os
ws = Workspace.from_config()
experiment = Experiment(workspace=ws, name='logging-api-test')
# make a folder using the double dot to indicate parent folder
directory = '../logging-api'
if not os.path.exists(directory):
os.mkdir(directory)
# start a run
run = experiment.start_logging(outputs=directory)
# write a file to watched output folder
file_name = os.path.join(directory, "myfile.txt")
with open(file_name, "w") as f:
f.write('This is an output file that will be uploaded.\n')
# this will fail!
run2.complete()
```
stack trace
```
AzureMLAggregatedException Traceback (most recent call last)
in
----> 1 run3.complete()
~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/core/run.py in complete(self, _set_status)
1341 :type _set_status: bool
1342 """
-> 1343 self._client.complete(_set_status=_set_status)
1344
1345 def fail(self, error_details=None, error_code=None, _set_status=True):
~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_run_impl/run_history_facade.py in complete(self, _set_status)
592 """
593 self.flush()
--> 594 self.upload_tracked_files()
595 self.set_completed_status(_set_status=_set_status)
596
~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_run_impl/run_history_facade.py in upload_tracked_files(self)
658
659 def upload_tracked_files(self):
--> 660 self.output_file_tracker.upload_tracked_files()
661
662 def take_snapshot(self, file_or_folder_path, _raise_on_validation_failure=True):
~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_history/utils/context_managers.py in upload_tracked_files(self)
151
152 def upload_tracked_files(self):
--> 153 self.py_wd.track(self.artifacts_client, self.container_id, self.trackfolders, self.deny_list)
154
155
~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_history/utils/context_managers.py in track(self, artifacts_client, container_id, track_folders, blacklist)
93 for fs in self.fs_list:
94 self.logger.debug("Calling track for {}".format(fs.ident()))
---> 95 fs.track(artifacts_client, container_id, track_folders, blacklist)
96 return True
97
~/opt/miniconda3/envs/rvo/lib/python3.8/site-packages/azureml/_history/utils/filesystem.py in track(self, artifacts_client, container_id, track_folders, blacklist)
79
80 if len(exception_messages) > 0:
---> 81 raise AzureMLAggregatedException(exception_messages)
82
83 def _upload_folder(self, artifacts_client, container_id, path, blacklist):
AzureMLAggregatedException: AzureMLAggregatedException:
Message: Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.
StorageAccount: sasetfdevdsresvalue
ContainerName: azureml
StatusCode: 403
InnerException None
ErrorResponse
{
"error": {
"message": "Encountered authorization error while uploading to blob storage. Please check the storage account attached to your workspace. Make sure that the current user is authorized to access the storage account and that the request is not blocked by a firewall, virtual network, or other security setting.\n\tStorageAccount: sasetfdevdsresvalue\n\tContainerName: azureml\n\tStatusCode: 403"
}
}
```
貢獻指南
這個儲存庫沒有索引到貢獻指南
研究方向
從 issue 中連結的可重現 logging-api notebook 開始,追蹤 Run.complete() 經過 upload_tracked_files() 的執行流程。使用包含 '..' 的 outputs 目錄重現該失敗;當被追蹤的檔案成功上傳,且完成操作不再回報授權錯誤時,即表示完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- azure, jupyter-notebook, python
- 領域
- cloud, machine-learning
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100