Azure / Azure/azure-powershell
Set-AzDataFactoryV2Pipeline - pipelineReturnValue deserialise value json and set setSystemVariable
- Dominant language
- C#
- Stars
- 4.8k
- Forks
- 4.3k
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 51
Description
### Description
cmdlet Set-AzDataFactoryV2Pipeline does not deserialize the json structure contained in the value attribute of a SetVariable activity where the variableName is "pipelineReturnValue" as is configured when setting the variable type to pipeline return value.
In addition the setSystemVariable attribute is not published resulting in a "Variable in Set Variable Does Not Exists" error
# Configured through UI
```json
{
"name": "Example",
"properties": {
"activities": [
{
"name": "Set SomeValue",
"type": "SetVariable",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"variableName": "pipelineReturnValue",
"value": [
{
"key": "SomeValue",
"value": {
"type": "String",
"content": "FooBar"
}
}
],
"setSystemVariable": true
}
}
],
"parameters": {},
"variables": {},
"annotations": []
},
"type": "Microsoft.DataFactory/factories/pipelines"
}
```
# Published via Set-AzDataFactoryV2Pipeline
```json
{
"name": "Example",
"properties": {
"activities": [
{
"name": "Set SomeValue",
"type": "SetVariable",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"variableName": "pipelineReturnValue",
"value": "[{\"key\":\"SomeValue\",\"value\":{\"type\":\"String\",\"content\":\"FooBar\"}}]"
}
}
],
"annotations": [],
"lastPublishTime": "2023-05-02T02:08:29Z"
},
"type": "Microsoft.DataFactory/factories/pipelines"
}
```
### Issue script & Debug output
```PowerShell
DEBUG: 12:08:28 PM - SetAzureDataFactoryPipelineCommand begin processing with ParameterSet 'ByFactoryName'.
DEBUG: 12:08:28 PM - using account id '###############'...
DEBUG: 12:08:28 PM - [ConfigManager] Got nothing from [DisplayBreakingChangeWarning], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: [Common.Authentication]: Authenticating using Account: '###############', environment: 'AzureCloud', tenant: '###############'
DEBUG: 12:08:28 PM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'###############', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'###############'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95] Found 1 cache accounts and 0 broker accounts
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95] Returning 1 accounts
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] MSAL MSAL.NetCore with assembly version '4.39.0.0'. CorrelationId(###############)
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] === AcquireTokenSilent Parameters ===
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] LoginHint provided: False
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] Account provided: True
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] ForceRefresh: False
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############]
=== Request Data ===
Authority Provided? - True
Scopes - https://management.core.windows.net//.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - ###############
UserAssertion set: False
LongRunningOboCacheKey set: False
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.95 - ###############] === Token Acquisition (SilentRequest) started:
Scopes: https://management.core.windows.net//.default
Authority Host: login.microsoftonline.com
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.97 - ###############] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.97 - ###############] Access token is not expired. Returning the found cache entry. [Current time (05/02/2023 02:08:28) - Expiration Time (05/02/2023 03:19:39 +00:00) - Extended Expiration Time (05/02/2023 03:19:39 +00:00)]
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.97 - ###############] Returning access token found in cache. RefreshOn exists ? False
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.97 - ###############] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.97 - ###############] Fetched access token from host login.microsoftonline.com.
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.97 - ###############]
=== Token Acquisition finished successfully:
DEBUG: False MSAL 4.39.0.0 MSAL.NetCore .NET 7.0.0 Microsoft Windows 10.0.16299 [05/02 02:08:28.97 - ###############] AT expiration time: 2/05/2023 3:19:39 AM +00:00, scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default source Cache from login.microsoftonline.com appHashCode ###############
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId: ExpiresOn: 2023-05-02T03:19:39.0000000+00:00
DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: '###############', UserId: '###############'
DEBUG: ============================ HTTP REQUEST ============================
HTTP Method:
PUT
Absolute Uri:
https://management.azure.com/subscriptions/###############/resourceGroups/###############/providers/Microsoft.DataFactory/factories/###############/pipelines/Example?api-version=2018-06-01
Headers:
Accept-Language : en-US
x-ms-client-request-id : ###############
Body:
{
"properties": {
"activities": [
{
"type": "SetVariable",
"typeProperties": {
"variableName": "pipelineReturnValue",
"value": [
{
"key": "SomeValue",
"value": {
"type": "String",
"content": "FooBar"
}
}
]
},
"name": "Set SomeValue",
"dependsOn": [],
"userProperties": []
}
],
"parameters": {},
"variables": {},
"annotations": []
}
}
DEBUG: ============================ HTTP RESPONSE ============================
Status Code:
OK
Headers:
Cache-Control : no-cache
Pragma : no-cache
x-ms-correlation-request-id : ###############
Server : Kestrel
x-ms-ratelimit-remaining-subscription-writes: 1199
x-ms-request-id : ###############
x-ms-routing-request-id : ###############
Strict-Transport-Security : max-age=31536000; includeSubDomains
X-Content-Type-Options : nosniff
Date : Tue, 02 May 2023 02:08:29 GMT
Body:
{
"id": "/subscriptions/###############/resourceGroups/###############/providers/Microsoft.DataFactory/factories/###############/pipelines/Example",
"name": "Example",
"type": "Microsoft.DataFactory/factories/pipelines",
"properties": {
"activities": [
{
"type": "SetVariable",
"typeProperties": {
"variableName": "pipelineReturnValue",
"value": [
{
"key": "SomeValue",
"value": {
"type": "String",
"content": "FooBar"
}
}
]
},
"name": "Set SomeValue",
"dependsOn": [],
"userProperties": []
}
],
"parameters": {},
"variables": {},
"annotations": []
},
"etag": "3e005050-0000-1a00-0000-6450709d0000"
}
DEBUG: 12:08:29 PM - [ConfigManager] Got nothing from [EnableDataCollection], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: AzureQoSEvent: Module: Az.DataFactory:1.16.8; CommandName: Set-AzDataFactoryV2Pipeline; PSVersion: 7.3.1; IsSuccess: True; Duration: 00:00:00.7348217
DEBUG: Finish sending metric.
DEBUG: 12:08:29 PM - SetAzureDataFactoryPipelineCommand end processing.
```
### Environment data
```PowerShell
Name Value
---- -----
PSVersion 7.3.1
PSEdition Core
GitCommitId 7.3.1
OS Microsoft Windows 10.0.16299
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
```
### Module versions
```PowerShell
ModuleType Version PreRelease Name ExportedCommands
---------- ------- ---------- ---- ----------------
Script 2.9.0 Az.Accounts {Add-AzEnvironment, Clear-AzConfig, Clear-AzContext, Clear-AzDefault…}
Script 1.16.8 Az.DataFactory {Add-AzDataFactoryV2DataFlowDebugSessionPackage, Add-AzDataFactoryV2TriggerSubscription, Get-AzDataFactory, Get-AzDataFactoryActivityWindow…}
Script 0.110.1 azure.datafactory.tools {Export-AdfToArmTemplate, Get-AdfDocDiagram, Get-AdfFromService, Import-AdfFromFolder…}
```
### Error output
```PowerShell
N/A
```
Contributor guide
Assessment
This issue has not been assessed yet.