Azure / Azure/MachineLearningNotebooks

Pipeline parameters with '=' character in string value raise parsing exceptions

Open
#1,592 2 comments 0 reactions 0 assignees View on GitHub
ADO bug Pipelines
Dominant language
Jupyter Notebook
Stars
4.4k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

I have encountered this issue in different components (pipeline parameters, Databricks spark config values, etc.), the underlying key/value parsing mechanism seems not to support string values with '=' character:

```bash
[2021-09-14 09:37:34Z] Parsed interface. Will be submitting job to Azure Databricks adb-***.azuredatabricks.net
[2021-09-14 09:37:35Z] Failed to submit job because of error: Invalid base_parameters parameter some_parameter=AT_PW=14_FW=60_batch, stack_trace: at Microsoft.Aether.Clouds.CloudCommon.ParameterParser.TryGetDictionary(Dictionary`2 parameters, String name) in d:\dbs\sh\l0hi\0902_133310\cmd\2h\src\aether\platform\backendV2\Clouds\Common\CloudCommon\ParameterParser.cs:line 41
at Microsoft.Aether.DataBricksCloud.K8sService.Parser.ValidateWorkloadSetup(DataBricksJobMetadata job, Dictionary`2 parameters) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\Parser.cs:line 427
at Microsoft.Aether.DataBricksCloud.K8sService.Parser.GetSubmitRunDto(DataBricksJobMetadata job, Dictionary`2 metadata, Dictionary`2 parameters) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\Parser.cs:line 31
at Microsoft.Aether.DataBricksCloud.K8sService.JobProcessor.StartNewJobAsync(DataBricksJobMetadata job) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\JobProcessor.cs:line 264
[2021-09-14 09:37:35Z] Exception while running job, error: Invalid base_parameters parameter some_parameter=AT_PW=14_FW=60_batch, stack_trace: at Microsoft.Aether.Clouds.CloudCommon.ParameterParser.TryGetDictionary(Dictionary`2 parameters, String name) in d:\dbs\sh\l0hi\0902_133310\cmd\2h\src\aether\platform\backendV2\Clouds\Common\CloudCommon\ParameterParser.cs:line 41
at Microsoft.Aether.DataBricksCloud.K8sService.Parser.ValidateWorkloadSetup(DataBricksJobMetadata job, Dictionary`2 parameters) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\Parser.cs:line 427
at Microsoft.Aether.DataBricksCloud.K8sService.Parser.GetSubmitRunDto(DataBricksJobMetadata job, Dictionary`2 metadata, Dictionary`2 parameters) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\Parser.cs:line 31
at Microsoft.Aether.DataBricksCloud.K8sService.JobProcessor.StartNewJobAsync(DataBricksJobMetadata job) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\JobProcessor.cs:line 264
at Microsoft.Aether.DataBricksCloud.K8sService.JobProcessor.ProcessJobAsync(DataBricksJobMetadata job) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\JobProcessor.cs:line 188
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.