Azure / Azure/azureml-examples

Changing output type to "uri_file" doesn't work

Open
#1,374 0 comments 0 reactions 0 assignees View on GitHub
example issue pipeline
Dominant language
Jupyter Notebook
Stars
2k
Forks
1.7k
Avg merge
18h 18m
Merged PRs (30d)
2

Description

## Which example? Describe the issue

example: https://github.com/Azure/azureml-examples/blob/sdk-preview/sdk/jobs/pipelines/2e_image_classification_keras_minist_convnet/prep/prep_component.py#L19-L20
description: I changed `Output(type="uri_folder")` to `Output(type="uri_file")` (and updating https://github.com/Azure/azureml-examples/blob/sdk-preview/sdk/jobs/pipelines/2e_image_classification_keras_minist_convnet/prep/prep_component.py#L25 and https://github.com/Azure/azureml-examples/blob/sdk-preview/sdk/jobs/pipelines/2e_image_classification_keras_minist_convnet/prep/prep_component.py#L31 to `training_data,` and `test_data,`, respectively). However, I get the below error, because `training_data` and `test_data` still point to folders, not files:

```
{'code': ExecutionFailed, 'message': [{"exit_code":1,"error_message":"Execution failed with error: [stderr]Traceback (most recent call last):\n[stderr] File \"/azureml-envs/azureml_7b92ddcb730b2da53d9de4d730d256dd/lib/python3.7/runpy.py\", line 193, in _run_module_as_main\n[stderr] \"__main__\", mod_spec)\n[stderr] File \"/azureml-envs/azureml_7b92ddcb730b2da53d9de4d730d256dd/lib/python3.7/runpy.py\", line 85, in _run_code\n[stderr] exec(code, run_globals)\n[stderr] File \"/azureml-envs/azureml_7b92ddcb730b2da53d9de4d730d256dd/lib/python3.7/site-packages/mldesigner/executor.py\", line 75, in \n[stderr] ).execute(func_args)\n[stderr] File \"/azureml-envs/azureml_7b92ddcb730b2da53d9de4d730d256dd/lib/python3.7/site-packages/mldesigner/_component_executor.py\", line 203, in execute\n[stderr] run = self._func(**args)\n[stderr] File \"/mnt/azureml/cr/j/490b3ff88ae54e7a8c82925c2e94e718/exe/wd/prep_component.py\", line 26, in prepare_data_component\n[stderr] 60000,\n[stderr] File \"/mnt/azureml/cr/j/490b3ff88ae54e7a8c82925c2e94e718/exe/wd/prep_component.py\", line 39, in convert\n[stderr] o = open(outf, \"w\")\n[stderr]IsADirectoryError: [Errno 21] Is a directory: '/mnt/azureml/cr/j/490b3ff88ae54e7a8c82925c2e94e718/cap/data-capability/wd/training_data'\n","process_name":"python","error_file":"user_logs/std_log.txt"}], 'target': , 'category': UserError, 'error_details': [{'key': exit_codes, 'value': 1}, ], 'inner_error': null}
```

i would expect `training_data` and `test_data` to either point to (1) writeable locations that don't yet exist, or (2) empty dummy files that i can overwrite. I don't want it to point to a folder that already exists, because that means I can't write to that path.

Can I not use implicit intermediate files between nodes? Do I always have to use folders?

## Additional context

I can successfully use a "uri_file"-type input.

Contributor guide

Open the contributing guide

Research direction

Start with sdk/jobs/pipelines/2e_image_classification_keras_minist_convnet/prep/prep_component.py, especially lines 19-20, 25, and 31, and reproduce the uri_file output failure. Read the traceback around prepare_data_component and convert, then verify the expected intermediate-output behavior; done means the example works with uri_file outputs or clearly documents the supported limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.