Azure / Azure/MachineLearningNotebooks
Pipeline parameters with '=' character in string value raise parsing exceptions
- Vorherrschende Sprache
- Jupyter Notebook
- Sterne
- 4.4k
- Forks
- 2.6k
- PR-Merge-Kennzahlen
- Keine gemergten PRs in 30 T.
Beschreibung
I have encountered this issue in different components (pipeline parameters, Databricks spark config values, etc.), the underlying key/value parsing mechanism seems not to support string values with '=' character:
```bash
[2021-09-14 09:37:34Z] Parsed interface. Will be submitting job to Azure Databricks adb-***.azuredatabricks.net
[2021-09-14 09:37:35Z] Failed to submit job because of error: Invalid base_parameters parameter some_parameter=AT_PW=14_FW=60_batch, stack_trace: at Microsoft.Aether.Clouds.CloudCommon.ParameterParser.TryGetDictionary(Dictionary`2 parameters, String name) in d:\dbs\sh\l0hi\0902_133310\cmd\2h\src\aether\platform\backendV2\Clouds\Common\CloudCommon\ParameterParser.cs:line 41
at Microsoft.Aether.DataBricksCloud.K8sService.Parser.ValidateWorkloadSetup(DataBricksJobMetadata job, Dictionary`2 parameters) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\Parser.cs:line 427
at Microsoft.Aether.DataBricksCloud.K8sService.Parser.GetSubmitRunDto(DataBricksJobMetadata job, Dictionary`2 metadata, Dictionary`2 parameters) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\Parser.cs:line 31
at Microsoft.Aether.DataBricksCloud.K8sService.JobProcessor.StartNewJobAsync(DataBricksJobMetadata job) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\JobProcessor.cs:line 264
[2021-09-14 09:37:35Z] Exception while running job, error: Invalid base_parameters parameter some_parameter=AT_PW=14_FW=60_batch, stack_trace: at Microsoft.Aether.Clouds.CloudCommon.ParameterParser.TryGetDictionary(Dictionary`2 parameters, String name) in d:\dbs\sh\l0hi\0902_133310\cmd\2h\src\aether\platform\backendV2\Clouds\Common\CloudCommon\ParameterParser.cs:line 41
at Microsoft.Aether.DataBricksCloud.K8sService.Parser.ValidateWorkloadSetup(DataBricksJobMetadata job, Dictionary`2 parameters) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\Parser.cs:line 427
at Microsoft.Aether.DataBricksCloud.K8sService.Parser.GetSubmitRunDto(DataBricksJobMetadata job, Dictionary`2 metadata, Dictionary`2 parameters) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\Parser.cs:line 31
at Microsoft.Aether.DataBricksCloud.K8sService.JobProcessor.StartNewJobAsync(DataBricksJobMetadata job) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\JobProcessor.cs:line 264
at Microsoft.Aether.DataBricksCloud.K8sService.JobProcessor.ProcessJobAsync(DataBricksJobMetadata job) in d:\dbs\sh\l0hi\0903_131236\cmd\i\src\aether\platform\backendV2\BlueBox\Clouds\DataBricksK8s\Service\JobProcessor.cs:line 188
```
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Rechercherichtung
Start with the ParameterParser.TryGetDictionary method at ParameterParser.cs line 41, identified in the stack trace, and trace how pipeline and Databricks parameter strings are split. Reproduce the shown value containing '=' and add coverage demonstrating that such values parse without an exception.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- azure
- Bereich
- cloud
- Issue-Typ
- Bug
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 35/100