[BUG] After AzureCli@2 uses azure-cli==2.70.0 instead of azure-cli==2.65.0, importing azure.ai.ml module has ImportError: cannot import name '_T' from 'marshmallow.fields'
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 3.5k
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 64
Description
### Describe the bug
When using AzureCli@2 task in Azure DevOps, if I run a python script, in which I import modules from azure.ai.ml, it will have the error below.
It was working before when AzureCLI@2 has the azure-cli version as 2.65.0. I ran the same pipeline without any changes last week and it worked. But this week, when the azure-cli version is 2.70.0, this bug occurs.
### Related command
- task: AzureCLI@2
inputs:
azureSubscription: ${{ parameters.WRKSP_SVC_CONN }}
scriptType: bash
scriptLocation: inlineScript
workingDirectory: $(Build.SourcesDirectory)
addSpnToEnvironment: true
inlineScript: |
python -m sample
Inside the sample.py file:
from azure.ai.ml import MLClient, Output, command, dsl
### Errors
```
File "/usr/local/lib/python3.11/site-packages/azure/ai/ml/entities/_assets/__init__.py", line 9, in
from ._artifacts.code import Code
File "/usr/local/lib/python3.11/site-packages/azure/ai/ml/entities/_assets/_artifacts/code.py", line 10, in
from azure.ai.ml._schema import CodeAssetSchema
File "/usr/local/lib/python3.11/site-packages/azure/ai/ml/_schema/__init__.py", line 7, in
from ._sweep import SweepJobSchema
File "/usr/local/lib/python3.11/site-packages/azure/ai/ml/_schema/_sweep/__init__.py", line 7, in
from .sweep_job import SweepJobSchema
File "/usr/local/lib/python3.11/site-packages/azure/ai/ml/_schema/_sweep/sweep_job.py", line 7, in
from azure.ai.ml._schema._sweep.parameterized_sweep import ParameterizedSweepSchema
File "/usr/local/lib/python3.11/site-packages/azure/ai/ml/_schema/_sweep/parameterized_sweep.py", line 7, in
from azure.ai.ml._schema.core.fields import ExperimentalField, NestedField, PathAwareSchema
File "/usr/local/lib/python3.11/site-packages/azure/ai/ml/_schema/core/fields.py", line 19, in
from marshmallow.fields import _T, Field, Nested
ImportError: cannot import name '_T' from 'marshmallow.fields' (/usr/local/lib/python3.11/site-packages/marshmallow/fields.py)
```
### Issue script & Debug output
Not applicable
### Expected behavior
The error should not happen, and the import should happen successfully.
### Environment Summary
azure-cli 2.70.0
core 2.70.0
telemetry 1.1.0
Dependencies:
msal 1.31.2b1
azure-mgmt-resource 23.1.1
Python location '/usr/local/bin/python3.11'
Config directory '/home/AzDevOps_azpcontainer/.azure'
Extensions directory '/home/AzDevOps_azpcontainer/.azure/cliextensions'
Python (Linux) 3.11.11 (main, Feb 25 2025, 05:21:12) [GCC 12.2.0]
### Additional context
For running this Azure Pipeline, I use custom image from Azure Container Registry, and in that image, I installed azure-ai-ml==1.25.0
I have no issue running imports from azure-ai-ml==1.25.0 elsewhere. This issue only happens when I run this python file in the AzureCli@2 task. And I have to use AzureCli@2 because it can addSpnToEnvironment.
I have found a hotfix solution: I have to manually add this to every single AzureCli@2 task if I run a python file with azure.ai.ml imports in it:
`python -m pip install marshmallow==3.23.2`
It resolves the error, but this is not ideal at all. Please address this.
Contributor guide
Research direction
Reproduce the AzureCli@2 scenario with sample.py importing azure.ai.ml on azure-cli 2.70.0, then inspect how its bundled dependencies select marshmallow alongside azure-ai-ml==1.25.0. Done means the import succeeds without manually installing marshmallow==3.23.2, with coverage for the affected dependency combination.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100