Azure / Azure/azure-powershell

Set-AzSynapsePipeline error out when pipeline definition has multiple "boolean" fields being set using Expression type.

Open
#20,881 7 comments 0 reactions 0 assignees View on GitHub
act-codegen-extensibility-squad bug Service Attention Synapse
Dominant language
C#
Stars
4.8k
Forks
4.3k
Avg merge
2d 17h
Merged PRs (30d)
51

Description

### Description

When invoking "Set-AzSynapsePipeline" to deploy an Synapse pipeline, if there are multiple "boolean" properties that are using Expression type, the cmdlet failed to deploy.

![image](https://user-images.githubusercontent.com/48073090/217484141-8894809b-f8f5-4f26-b115-c2ef875a702c.png)

![image](https://user-images.githubusercontent.com/48073090/217484345-fd5a5dee-7f47-4f74-8c4e-4432d3b04fc8.png)

[test_pipeline_2022_01_03.zip](https://github.com/Azure/azure-powershell/files/10683700/test_pipeline_2022_01_03.zip)

### Issue script & Debug output

```PowerShell
Set-AzSynapsePipeline -WorkspaceName 'sandbox-xxxxxx' -Name 'test_pipeline_2022_01_03' -DefinitionFile "C:\Repo\xxxxxxx\test_pipeline_2022_01_03.json"
```

### Environment data

```PowerShell
Name Value
---- -----
PSVersion 7.3.2
PSEdition Core
GitCommitId 7.3.2
OS Microsoft Windows 10.0.22621
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
```

### Module versions

```PowerShell
ModuleType Version PreRelease Name ExportedCommands
---------- ------- ---------- ---- ----------------
Script 2.11.1 Az.Accounts {Add-AzEnvironment, Clear-AzConfig, Clear-AzContext, Clear-AzDefault…}
Script 2.2.0 Az.Synapse {Add-AzSynapseDataFlowDebugSessionPackage, Add-AzSynapseTriggerSubscription, Clear-AzSynapseSql…
```

### Error output

```PowerShell
Message : Exception has been thrown by the target of an invocation.
StackTrace : at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.Azure.Commands.Synapse.Common.JsonConvert.DeserializeObject[T](String rawJsonContent)
at Microsoft.Azure.Commands.Synapse.Models.SynapseAnalyticsArtifactsClient.CreateOrUpdatePipeline(String pipelineName, String rawJsonContent)
at Microsoft.Azure.Commands.Synapse.SetAzureSynapsePipeline.ExecuteCmdlet()
at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception : System.Reflection.TargetInvocationException
InvocationInfo : {Set-AzSynapsePipeline}
Line : Set-AzSynapsePipeline -WorkspaceName 'sandbox-xxxxxx' -Name 'test_pipeline_2022_01_03' -DefinitionFile
"C:\Repo\xxxxxxx\test_pipeline_2022_01_03.json"

Position : At line:2 char:1
+ Set-AzSynapsePipeline -WorkspaceName 'sandbox-xxxxxx' -N …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId : 61

Message : The requested operation requires an element of type 'Boolean', but the target element has type 'Object'.
StackTrace : at System.Text.Json.JsonElement.g__ThrowJsonElementWrongTypeException|17_0(JsonTokenType actualType)
at Azure.Analytics.Synapse.Artifacts.Models.AzureBlobFSReadSettings.DeserializeAzureBlobFSReadSettings(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.StoreReadSettings.DeserializeStoreReadSettings(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.ParquetSource.DeserializeParquetSource(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.CopySource.DeserializeCopySource(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.CopyActivity.DeserializeCopyActivity(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.Activity.DeserializeActivity(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.SwitchActivity.DeserializeSwitchActivity(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.Activity.DeserializeActivity(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.ForEachActivity.DeserializeForEachActivity(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.Activity.DeserializeActivity(JsonElement element)
at Azure.Analytics.Synapse.Artifacts.Models.PipelineResource.DeserializePipelineResource(JsonElement element)
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
Exception : System.InvalidOperationException
InvocationInfo : {Set-AzSynapsePipeline}
Line : Set-AzSynapsePipeline -WorkspaceName 'sandbox-xxxxxx' -Name 'test_pipeline_2022_01_03' -DefinitionFile
"C:\Repo\xxxxxxx\test_pipeline_2022_01_03.json"

Position : At line:2 char:1
+ Set-AzSynapsePipeline -WorkspaceName 'sandbox-xxxxxx' -N …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId : 61
```

Contributor guide

Open the contributing guide

Research direction

Reproduce Set-AzSynapsePipeline with the attached test_pipeline_2022_01_03.json, then trace Microsoft.Azure.Commands.Synapse.Common.JsonConvert.DeserializeObject through CreateOrUpdatePipeline and the generated Synapse model deserializers named in the stack trace. Done means a pipeline containing multiple boolean Expression fields deploys successfully without the Boolean/Object exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp, powershell
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.