az ml cli v2 pipeline yml does not support keyword 'is_deterministic'
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 3.5k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 60
Description
**Related command**
**Describe the bug**
I want to run an azure ml pipeline using azure-cli v2. The steps should be non-deterministic (in sdk = 'allow_reuse'=False). Based on the pipeline schema, this should be set using
```yaml
is_deterministic: false
```
which is not accepted when submitting the job using
```bash
az ml job create -f pipeline.yml --web
```
it throws an unrelated error:
```bash
Met error :ParameterizedParallel.__init__() got an unexpected keyword argument 'environment'
```
When submitting the job without setting deterministic, the pipeline works fine (but being not deterministic)
Here is the pipeline yml definition I use:
```yaml
$schema: https://azuremlschemas.azureedge.net/latest/pipelineJob.schema.json
type: pipeline
compute: azureml:cpu-cluster
jobs:
scrape:
code: ./src
command: python run.py --dataset_path ${{inputs.datainput}}
environment: azureml:my_environment@latest
inputs:
datainput:
type: uri_folder
path: azureml://datastores/workspaceblobstore/paths/path/to/my/folder/
is_deterministic: false # without this - pipeline works fine (not deterministic)
```
**To Reproduce**
Create any pipeline with constant input parameters and no explicit output. Try to make it deterministic using above yml file.
**Expected behavior**
Setting 'is_deterministic: false' should be a valid entry and error is **not** raised.
**Environment summary**
```bash
az version
```
outputs:
```bash
{
"azure-cli": "2.37.0",
"azure-cli-core": "2.37.0",
"azure-cli-telemetry": "1.0.6",
"extensions": {
"ml": "2.4.1"
}
}
```
Contributor guide
Assessment
This issue has not been assessed yet.