Azure / Azure/azure-sdk-for-python
Download of URI folder is broken in 1.35.0
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
- **Package Name**: azure-ai-ml
- **Package Version**: 1.35.0
- **Operating System**: macOS 26.6.2 (arm64); also reproduces in the `python:3.14-slim` (Linux) container used in production
- **Python Version**: 3.14.6
**Describe the bug**
`MLClient.jobs.download(name=job_name, output_name=)` silently downloads nothing for a named `uri_folder` output of a `PipelineJob`, without raising any error.
The root cause is in `JobOperations._get_named_output_uri()` → `get_job_output_uris_from_dataplane()` (`azure/ai/ml/operations/_job_ops_helper.py`). That helper filters the RunHistory `run_metadata.outputs` dict to only the entries whose `.type` is a member of the SDK's own `DataType` enum:
```python
dataset_ids = [
run_outputs[output_name].asset_id
for output_name in output_names
if run_outputs[output_name].type in [o.value for o in DataType]
]
```
`DataType`'s values are lowercase-with-underscore (`"uri_file"`, `"uri_folder"`, `"mltable"`), but for a pipeline job's outputs, the RunHistory dataplane reports the wire type as PascalCase (`"UriFolder"`), e.g.:
```python
>>> client.jobs._runs_operations.get_run_data(job_name).run_metadata.outputs["forecast_data"].type
'UriFolder'
>>> 'UriFolder' in [o.value for o in DataType]
False
```
Because of this casing mismatch, every pipeline-job `uri_folder`/`uri_file`/`mltable` output is silently excluded from `dataset_ids`, `get_batch_dataset_uris()` is never called for it, and `_get_named_output_uri()` returns an empty dict for that output name. `JobOperations.download()` then just logs a debug message (`Could not download output ""...`) and returns having downloaded nothing — no exception, no visible signal that anything went wrong.
The underlying data is completely fine: fetching the asset id directly and resolving it through the dataset dataplane manually (bypassing the broken filter) downloads it without any problem:
```python
run_outputs = client.jobs._runs_operations.get_run_data(job_name).run_metadata.outputs
asset_id = run_outputs["forecast_data"].asset_id
uris = client.jobs._dataset_dataplane_operations.get_batch_dataset_uris([asset_id])
uri = uris.values_property[asset_id].uri
# uri resolves fine and download_artifact_from_aml_uri(uri=uri, ...) downloads the folder successfully
```
**To Reproduce**
1. Deploy a `PipelineJob` (e.g. via a `pipelineComponentBatchDeployment` behind a batch endpoint, or any multi-step pipeline) with at least one `uri_folder` output at the top level (e.g. `outputs = {"my_output": Output(type="uri_folder")}`).
2. Run the job to completion (`status == "Completed"`).
3. Call:
```python
client.jobs.download(name=job_name, download_path="./out", output_name="my_output")
```
4. Observe that `./out` is empty (or only contains unrelated artifacts), with no exception raised.
5. To confirm the root cause directly:
```python
run_outputs = client.jobs._runs_operations.get_run_data(job_name).run_metadata.outputs
print(run_outputs["my_output"].type) # -> "UriFolder"
from azure.ai.ml._restclient.arm_ml_service.models import DataType
print(run_outputs["my_output"].type in [o.value for o in DataType]) # -> False
```
**Expected behavior**
`client.jobs.download(output_name="my_output")` should download the named output's contents, matching the output's actual type regardless of casing (or at minimum raise a clear error instead of silently succeeding with nothing downloaded). `get_job_output_uris_from_dataplane()`'s type filter should compare the wire value case/format-insensitively (e.g. normalize both sides, such as `.lower().replace("_", "")`), or use the `DataType` enum's actual wire values instead of the Python-side `.value` strings.
**Screenshots**
N/A
**Additional context**
- This behaves correctly in `azure-ai-ml` 1.34.1, where the same call to `jobs.download(output_name=...)` for the same kind of pipeline job downloads correctly. Something in the 1.35.0 migration (changelog: "Migrated SDK entities and their consumers off the per-version msrest REST clients onto the shared `arm_ml_service` hybrid client") appears to have introduced or exposed this casing mismatch between the wire type and the `DataType` enum.
- This is especially dangerous because it fails *silently* — no exception is raised, so callers have no signal that the download produced nothing unless they separately verify the expected files exist on disk afterward.
- Confirmed against a real, long-completed batch-endpoint pipeline job in our workspace; the underlying blob data was present and fully intact the entire time — only the SDK's output resolution was broken.
- Workaround we're using in application code: resolve the output's asset id via `client.jobs._runs_operations.get_run_data(job_name).run_metadata.outputs[output_name].asset_id`, resolve it to a URI via `client.jobs._dataset_dataplane_operations.get_batch_dataset_uris([asset_id])`, and download that URI with `azure.ai.ml._artifacts._artifact_utilities.download_artifact_from_aml_uri`. This relies on private/internal APIs and should not be necessary.
Contributor guide
Assessment
This issue has not been assessed yet.