Azure / Azure/azure-cli

az ml cli v2 pipeline yml does not support keyword 'is_deterministic'

Open
#23,406 7 comments 0 reactions 0 assignees View on GitHub
act-codegen-extensibility-squad Auto-Assign customer-reported Machine Learning Service Attention
Dominant language
Python
Stars
4.6k
Forks
3.5k
Avg merge
3d 2h
Merged PRs (30d)
60

Description

**Related command**

**Describe the bug**
I want to run an azure ml pipeline using azure-cli v2. The steps should be non-deterministic (in sdk = 'allow_reuse'=False). Based on the pipeline schema, this should be set using
```yaml
is_deterministic: false
```
which is not accepted when submitting the job using
```bash
az ml job create -f pipeline.yml --web
```
it throws an unrelated error:
```bash
Met error :ParameterizedParallel.__init__() got an unexpected keyword argument 'environment'
```
When submitting the job without setting deterministic, the pipeline works fine (but being not deterministic)
Here is the pipeline yml definition I use:

```yaml
$schema: https://azuremlschemas.azureedge.net/latest/pipelineJob.schema.json
type: pipeline
compute: azureml:cpu-cluster
jobs:
scrape:
code: ./src
command: python run.py --dataset_path ${{inputs.datainput}}
environment: azureml:my_environment@latest
inputs:
datainput:
type: uri_folder
path: azureml://datastores/workspaceblobstore/paths/path/to/my/folder/
is_deterministic: false # without this - pipeline works fine (not deterministic)
```

**To Reproduce**
Create any pipeline with constant input parameters and no explicit output. Try to make it deterministic using above yml file.

**Expected behavior**
Setting 'is_deterministic: false' should be a valid entry and error is **not** raised.

**Environment summary**
```bash
az version
```
outputs:
```bash
{
"azure-cli": "2.37.0",
"azure-cli-core": "2.37.0",
"azure-cli-telemetry": "1.0.6",
"extensions": {
"ml": "2.4.1"
}
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.