Azure / Azure/azure-powershell

Issue with creating a Stream Analytics job output to Data Explorer

Open
#23,260 3 comments 0 reactions 0 assignees View on GitHub
bug Service Attention Stream Analytics Tracking
Dominant language
C#
Stars
4.8k
Forks
4.3k
Avg merge
2d 17h
Merged PRs (30d)
51

Description

### Description

I'm experiencing an issue while trying to create a Stream Analytics job output to Data Explorer using the `New-AzStreamAnalyticsOutput` cmdlet in PowerShell. The data source JSON I'm using for the configuration is as follows:

```json
{
"name": "test123",
"type": "Microsoft.StreamAnalytics/streamingjobs/outputs",
"properties": {
"datasource": {
"properties": {
"database": "databaseName",
"table": "tableName",
"authenticationMode": "Msi",
"cluster": "https://xyz.westeurope.kusto.windows.net"
},
"type": "Microsoft.Kusto/clusters/databases"
},
"serialization": {
"properties": {
"encoding": "UTF8"
},
"type": "Json"
}
}
}
```

However, when I run the cmdlet, I receive the following error:

```
New-AzStreamAnalyticsOutput_Create: C:\Users\YourUser\Documents\PowerShell\Modules\Az.StreamAnalytics\2.0.0\exports\ProxyCmdletDefinitions.ps1:4505:9
Line |
4505 | $steppablePipeline.Begin($PSCmdlet)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| The JSON provided in the request body is invalid. Required property 'properties' not found in JSON. Path '', line 9, position 19.
```

I'm not sure what I'm doing wrong here. I noticed that the Az.StreamAnalytics 2.0.0 was published on 2021-05-25. However, the latest non-preview ARM rest endpoint for Stream Analytics is from 2020-03-01. Could it be that creating a Data Explorer output simply does not work yet with Az.StreamAnalytics 2.0.0?

Any help or guidance would be greatly appreciated. Thanks!

### Issue script & Debug output

```PowerShell
$fileName = '.\output.json'
ConvertTo-Json $outputObject -Depth 10 | Out-File -FilePath $fileName

$streamAnalyticsOutput = New-AzStreamAnalyticsOutput -ResourceGroupName $ResourceGroupName `
-JobName $StreamAnalyticsJobName `
-Name $OutputName `
-File $fileName
```

### Environment data

```PowerShell
Name Value
---- -----
PSVersion 7.3.9
PSEdition Core
GitCommitId 7.3.9
OS Microsoft Windows 10.0.22621
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
```

### Module versions

```PowerShell
ModuleType Version PreRelease Name
---------- ------- ---------- ----
Script 10.3.0 Az
Script 2.13.1 Az.Accounts
Script 2.0.0 Az.Advisor
Script 5.5.1 Az.Aks
Script 1.1.4 Az.AnalysisServices
Script 4.0.2 Az.ApiManagement
Script 1.3.0 Az.AppConfiguration
Script 2.2.2 Az.ApplicationInsights
Script 1.0.0 Az.ArcResourceBridge
Script 2.0.0 Az.Attestation
Script 1.0.0 Az.Automanage
Script 1.9.1 Az.Automation
Script 3.5.0 Az.Batch
Script 2.0.3 Az.Billing
Script 3.1.0 Az.Cdn
Script 1.2.0 Az.CloudService
Script 1.14.0 Az.CognitiveServices
Script 6.3.0 Az.Compute
Script 1.0.0 Az.ConfidentialLedger
Script 3.2.1 Az.ContainerInstance
Script 4.1.1 Az.ContainerRegistry
Script 1.12.0 Az.CosmosDB
Script 0.1.1 Az.Dashboard
Script 1.1.0 Az.DataBoxEdge
Script 1.7.0 Az.Databricks
Script 1.17.0 Az.DataFactory
Script 1.0.3 Az.DataLakeAnalytics
Script 1.3.0 Az.DataLakeStore
Script 2.0.0 Az.DataProtection
Script 1.0.1 Az.DataShare
Script 1.1.0 Az.DeploymentManager
Script 4.0.0 Az.DesktopVirtualization
Script 1.0.2 Az.DevTestLabs
Script 1.1.3 Az.Dns
Script 1.6.0 Az.EventGrid
Script 4.1.0 Az.EventHub
Script 1.10.0 Az.FrontDoor
Script 4.0.6 Az.Functions
Script 6.0.1 Az.HDInsight
Script 2.0.0 Az.HealthcareApis
Script 2.7.5 Az.IotHub
Script 4.11.0 Az.KeyVault
Script 2.2.0 Az.Kusto
Script 1.0.0 Az.LoadTesting
Script 1.5.0 Az.LogicApp
Script 1.1.3 Az.MachineLearning
Script 1.0.0 Az.MachineLearningServices
Script 1.3.1 Az.Maintenance
Script 1.1.1 Az.ManagedServiceIdentity
Script 3.0.0 Az.ManagedServices
Script 2.0.0 Az.MarketplaceOrdering
Script 1.1.2 Az.Media
Script 2.2.0 Az.Migrate
Script 4.6.0 Az.Monitor
Script 1.1.1 Az.MySql
Script 6.2.0 Az.Network
Script 1.1.2 Az.NotificationHubs
Script 3.2.0 Az.OperationalInsights
Script 1.6.3 Az.PolicyInsights
Script 1.1.0 Az.PostgreSql
Script 1.2.1 Az.PowerBIEmbedded
Script 1.0.4 Az.PrivateDns
Script 6.5.1 Az.RecoveryServices
Script 1.8.0 Az.RedisCache
Script 1.2.0 Az.RedisEnterpriseCache
Script 2.0.0 Az.Relay
Script 1.1.0 Az.ResourceMover
Script 6.10.0 Az.Resources
Script 1.4.0 Az.Security
Script 3.0.2 Az.SecurityInsights
Script 3.0.0 Az.ServiceBus
Script 3.2.0 Az.ServiceFabric
Script 2.0.0 Az.SignalR
Script 4.10.0 Az.Sql
Script 2.1.0 Az.SqlVirtualMachine
Script 2.1.2 Az.StackHCI
Script 5.10.0 Az.Storage
Script 1.0.1 Az.StorageMover
Script 2.0.0 Az.StorageSync
Script 2.0.0 Az.StreamAnalytics
Script 1.0.0 Az.Support
Script 3.0.3 Az.Synapse
Script 1.2.1 Az.TrafficManager
Script 3.1.1 Az.Websites
```

### Error output

```PowerShell
Message : The JSON provided in the request body is invalid. Required property 'properties' not found in JSON. Path '', line 9, position 19.
StackTrace : at Microsoft.Azure.PowerShell.Cmdlets.StreamAnalytics.StreamAnalytics.OutputsCreateOrReplace_Call(HttpRequestMessage request, Func`3 onOk, Func`3 onCreated, IEventListener eventListener, ISendAsync sender)
at Microsoft.Azure.PowerShell.Cmdlets.StreamAnalytics.StreamAnalytics.OutputsCreateOrReplace_Call(HttpRequestMessage request, Func`3 onOk, Func`3 onCreated, IEventListener eventListener, ISendAsync sender)
at Microsoft.Azure.PowerShell.Cmdlets.StreamAnalytics.StreamAnalytics.OutputsCreateOrReplace(String ifMatch, String ifNoneMatch, String subscriptionId, String resourceGroupName, String jobName, String outputName, IOutput body, Func`3 onOk, Func`3 onCreated, IEventListener eventListener, ISendAsync sender)
at Microsoft.Azure.PowerShell.Cmdlets.StreamAnalytics.Cmdlets.NewAzStreamAnalyticsOutput_Create.ProcessRecordAsync()
Exception : Microsoft.Azure.PowerShell.Cmdlets.StreamAnalytics.Runtime.UndeclaredResponseException
InvocationInfo : {New-AzStreamAnalyticsOutput_Create}
Line : $steppablePipeline.Begin($PSCmdlet)

Position : At C:\Users\AlexanderHuber\Documents\PowerShell\Modules\Az.StreamAnalytics\2.0.0\exports\ProxyCmdletDefinitions.ps1:4505 char:9
+ $steppablePipeline.Begin($PSCmdlet)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId : 13
```

Contributor guide

Open the contributing guide

Research direction

Start with the New-AzStreamAnalyticsOutput entry point and the generated ProxyCmdletDefinitions.ps1 location shown in the error. Compare the supplied output.json structure with the cmdlet's expected request body and investigate the Stream Analytics output creation call. Done means the documented Data Explorer configuration is accepted without the missing-properties error.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell
Domain
cli, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.