Azure / Azure/azure-powershell

Set-AzDataCollectionRule always fails when Updating a DataCollectionRule

Open
#19,381 3 comments 1 reaction 1 assignee Claimed by @VeryEarly View on GitHub
act-observability-squad Monitor question
Dominant language
C#
Stars
4.8k
Forks
4.3k
Avg merge
3d 14h
Merged PRs (30d)
54

Description

### Description

Set-AzDataCollectionRule always fails when Updating a DataCollectionRule
Tried from cloudshell as well

### Issue script & Debug output

```PowerShell
$DCRid="/subscriptions/3f954a4a-8cbf-448d-b855-bcafd4db20f1/resourceGroups/CTTestingRGNE44702/providers/Microsoft.Insights/dataCollectionRules/ct-dcrcfeb4c16-2d1e-4d25-802d-eb543b75052e"
$dcr = Get-AzDataCollectionRule -RuleId $DCRid
$dcr.Description="this is test"'
$dcr | Set-AzDataCollectionRule

PS /home/mayank> $dcr | Set-AzDataCollectionRule DEBUG: 8:26:41 AM - SetAzureRmDataCollectionRuleCommand begin processing with ParameterSet 'ByInputObject'.
DEBUG: 8:26:41 AM - using account id 'MSI@50342'...
DEBUG: 8:26:41 AM - [ConfigManager] Got nothing from [DisplayBreakingChangeWarning], Module = [], Cmdlet = []. Returning default value [True].
WARNING: 8:26:41 AM - *** The namespace for all the model classes will change from Microsoft.Azure.Management.Monitor.Management.Models to Microsoft.Azure.Management.Monitor.Models in future releases.
WARNING: 8:26:41 AM - *** The namespace for output classes will be uniform for all classes in future releases to make it independent of modifications in the model classes.
DEBUG: [Common.Authentication]: Authenticating using Account: 'MSI@50342', environment: 'AzureCloud', tenant: '72f988bf-86f1-41af-91ab-2d7cd011db47'
DEBUG: 8:26:41 AM - [ManagedServiceIdentityAuthenticator] Calling ManagedIdentityCredential.GetTokenAsync - TenantId:'72f988bf-86f1-41af-91ab-2d7cd011db47', Scopes:'https://management.core.windows.net/', UserId:''
DEBUG: ManagedIdentityCredential.GetToken invoked. Scopes: [ https://management.core.windows.net/ ] ParentRequestId:
DEBUG: Request [f1848214-33f8-4443-8447-cd0d794a1449] POST http://localhost:50342/oauth2/token
Metadata:REDACTED
x-ms-client-request-id:f1848214-33f8-4443-8447-cd0d794a1449
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Identity/1.6.0,(.NET 6.0.7; Linux 5.4.0-1089-azure #94~18.04.1-Ubuntu SMP Fri Aug 5 12:34:50 UTC 2022)
Content-Type:application/x-www-form-urlencoded
client assembly: Azure.Identity
DEBUG: Response [f1848214-33f8-4443-8447-cd0d794a1449] 200 OK (00.0s)
X-Powered-By:REDACTED
ETag:W/"920-1UL6jglpTnXG1mwdfw0eMjoWIHA"
Date:Fri, 02 Sep 2022 08:26:41 GMT
Connection:keep-alive
Keep-Alive:REDACTED
Content-Type:application/json; charset=utf-8
Content-Length:2336

DEBUG: ManagedIdentityCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net/ ] ParentRequestId: ExpiresOn: 2022-09-02T09:41:51.0000000+00:00
DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: '72f988bf-86f1-41af-91ab-2d7cd011db47', UserId: 'MSI@50342'
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
PUT

Absolute Uri:
https://management.azure.com/subscriptions/3f954a4a-8cbf-448d-b855-bcafd4db20f1/resourceGroups/cttestingrgne44702/providers/Microsoft.Insights/dataCollectionRules/ct-dcrcfeb4c16-2d1e-4d25-802d-eb543b75052e?api-version=2021-04-01

Headers:
x-ms-client-request-id : c43e014d-2d0c-4f46-943f-3d9ca98ac616
Accept-Language : en-US

Body:
{
"properties": {
"description": "Data collection rule for VM Insights health.",
"dataSources": {
"extensions": [
{
"streams": [
"Microsoft-ConfigurationChange"
],
"extensionName": "ChangeTracking-Windows",
"extensionSettings": {
"fileCollectionFrequency": 120,
"inventoryCollectionFrequency": 30000,
"softwareCollectionFrequency": 1800,
"serviceCollectionFrequency": 1800,
"registrySettings": {
"registryCollectionFrequency": 1800,
"registryInfo": [
{
"enabled": false,
"keyName": "HKEY_LOCAL_MACHINE\\Software\\Classes\\Directory\\Shellex\\CopyHookHandlers",
"valueName": "",
"recurse": true,
"groupTag": "",
"name": "DataSource_ChangeTrackingDefaultRegistry_1"
}
]
},
"fileSettings": {
"fileCollectionFrequency": 2700,
"fileinfo": [
{
"name": "ChangeTrackingCustomPath_as",
"enabled": true,
"path": "D:\\testfolder\\*",
"recurse": false,
"maxContentsReturnable": 5000000,
"maxOutputSize": 0,
"checksum": "Md5",
"pathType": "File",
"groupTag": "Custom"
}
]
}
},
"name": "CTDataSource-Windows"
},
{
"streams": [
"Microsoft-ConfigurationChange"
],
"extensionName": "ChangeTracking-Linux",
"extensionSettings": {
"fileCollectionFrequency": 120,
"softwareCollectionFrequency": 1800,
"serviceCollectionFrequency": 1800,
"inventoryCollectionFrequency": 30000,
"fileSettings": {
"fileCollectionFrequency": 2700,
"fileInfo": [
{
"name": "ChangeTrackingLinuxPath_ss",
"enabled": true,
"destinationPath": "/var/opt",
"useSudo": true,
"recurse": true,
"maxContentsReturnable": 0,
"pathType": "Directory",
"type": "Directory",
"links": "Follow",
"checksum": "Sha256",
"maxOutputSize": 5000000,
"groupTag": "Custom"
},
{
"name": "ChangeTrackingLinuxPath_default",
"enabled": true,
"destinationPath": "/etc/*.conf",
"useSudo": true,
"recurse": true,
"maxContentsReturnable": 5000000,
"pathType": "File",
"type": "File",
"links": "Follow",
"checksum": "",
"maxOutputSize": 500000,
"groupTag": "Recommended"
},
{
"name": "ChangeTrackingLinuxPath_test",
"checksum": "Sha256",
"destinationPath": "/home/testfolder/*.*",
"enabled": true,
"groupTag": "Custom",
"links": "Follow",
"maxContentsReturnable": 0,
"maxOutputSize": 5000000,
"pathType": "File",
"recurse": true,
"useSudo": true
}
]
}
},
"name": "CTDataSource-Linux"
}
]
},
"destinations": {
"logAnalytics": [
{
"workspaceResourceId": "/subscriptions/3f954a4a-8cbf-448d-b855-bcafd4db20f1/resourceGroups/CTTestingRGNE44702/providers/Microsoft.OperationalInsights/workspaces/LAWS-CT-TestNE44702"
}
],
"azureMonitorMetrics": {}
},
"dataFlows": [
{
"streams": [
"Microsoft-ConfigurationChange"
],
"destinations": [
"Microsoft-CT-Dest"
]
}
]
},
"location": "northeurope"
}

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
BadRequest

Headers:
Cache-Control : no-cache
Pragma : no-cache
x-ms-ratelimit-remaining-subscription-resource-requests: 149
Request-Context : appId=cid-v1:2bbfbac8-e1b0-44af-b9c6-3a40669d37e3
x-ms-correlation-request-id : 2c5007c5-b176-4507-88f2-a7af198e8716
x-ms-client-request-id : c43e014d-2d0c-4f46-943f-3d9ca98ac616
x-ms-routing-request-id : JIOINDIAWEST:20220902T082644Z:2c5007c5-b176-4507-88f2-a7af198e8716
x-ms-request-id : 88ce79aa-453a-49de-ae82-309770deff53
api-supported-versions : 2019-11-01-preview, 2021-04-01, 2021-09-01-preview, 2022-06-01
Server : Microsoft-HTTPAPI/2.0
Strict-Transport-Security : max-age=31536000; includeSubDomains
X-Content-Type-Options : nosniff
Date : Fri, 02 Sep 2022 08:26:45 GMT

Body:
{
"error": {
"code": "InvalidPayload",
"message": "Data collection rule is invalid",
"details": [
{
"code": "InvalidDestination",
"message": "'Destinations' destination names must be unique (case-insensitively). Duplicate names: .",
"target": "Properties.Destinations"
},
{
"code": "InvalidDataFlow",
"message": "'Data Flows' destination name is unknown: Microsoft-CT-Dest.",
"target": "Properties.DataFlows[0]"
}
]
}
}

DEBUG: 8:26:45 AM - [ConfigManager] Got [True] from [EnableDataCollection], Module = [], Cmdlet = [].
Set-AzDataCollectionRule: Exception type: ErrorResponseCommonV2Exception, Message: Microsoft.Azure.Management.Monitor.Models.ErrorResponseCommonV2Exception: Operation returned an invalid status code 'BadRequest'
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperations.CreateWithHttpMessagesAsync(String resourceGroupName, String dataCollectionRuleName, DataCollectionRuleResource body, Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperationsExtensions.CreateAsync(IDataCollectionRulesOperations operations, String resourceGroupName, String dataCollectionRuleName, DataCollectionRuleResource body, CancellationToken cancellationToken)
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperationsExtensions.Create(IDataCollectionRulesOperations operations, String resourceGroupName, String dataCollectionRuleName, DataCollectionRuleResource body)
at Microsoft.Azure.Commands.Insights.DataCollectionRules.SetAzureRmDataCollectionRuleCommand.ReplaceDataCollectionRule()
at Microsoft.Azure.Commands.Insights.DataCollectionRules.SetAzureRmDataCollectionRuleCommand.ProcessRecordInternal()
at Microsoft.Azure.Commands.Insights.MonitorCmdletBase.ExecuteCmdlet(), Code: Null, Status code:Null, Reason phrase: Null
DEBUG: 8:26:45 AM - [ConfigManager] Got nothing from [DisplayBreakingChangeWarning], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: AzureQoSEvent: Module: Az.Monitor:3.0.1; CommandName: Set-AzDataCollectionRule; PSVersion: 7.2.5; IsSuccess: False; Duration: 00:00:03.6409711; Exception: Exception type: ErrorResponseCommonV2Exception, Message: Microsoft.Azure.Management.Monitor.Models.ErrorResponseCommonV2Exception: Operation returned an invalid status code 'BadRequest'
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperations.CreateWithHttpMessagesAsync(String resourceGroupName, String dataCollectionRuleName, DataCollectionRuleResource body, Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperationsExtensions.CreateAsync(IDataCollectionRulesOperations operations, String resourceGroupName, String dataCollectionRuleName, DataCollectionRuleResource body, CancellationToken cancellationToken)
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperationsExtensions.Create(IDataCollectionRulesOperations operations, String resourceGroupName, String dataCollectionRuleName, DataCollectionRuleResource body)
at Microsoft.Azure.Commands.Insights.DataCollectionRules.SetAzureRmDataCollectionRuleCommand.ReplaceDataCollectionRule()
at Microsoft.Azure.Commands.Insights.DataCollectionRules.SetAzureRmDataCollectionRuleCommand.ProcessRecordInternal()
at Microsoft.Azure.Commands.Insights.MonitorCmdletBase.ExecuteCmdlet(), Code: Null, Status code:Null, Reason phrase: Null;
DEBUG: Finish sending metric.
DEBUG: 8:26:45 AM - SetAzureRmDataCollectionRuleCommand end processing.
```

### Environment data

```PowerShell
PS /home/mayank> $PSVersionTable
Name Value
---- -----
PSVersion 7.2.5
PSEdition Core
GitCommitId 7.2.5
OS Linux 5.4.0-1089-azure #94~18.04.1-Ubuntu SMP Fri Aug 5 12:34:50 UTC 2022
Platform Unix
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
```

### Module versions

```PowerShell
PS /home/mayank> Get-Module Az*

ModuleType Version PreRelease Name ExportedCommands
---------- ------- ---------- ---- ----------------
Script 2.9.1 Az.Accounts {Add-AzEnvironment, Clear-AzConfig, Clear-A…
Script 4.30.0 Az.Compute {Add-AzImageDataDisk, Add-AzVhd, Add-AzVMAd…
Script 3.0.1 Az.Monitor {Add-AzAutoscaleSetting, Add-AzLogProfile, …
Script 4.19.0 Az.Network {Add-AzApplicationGatewayAuthenticationCert…
Script 6.1.0 Az.Resources {Export-AzResourceGroup, Export-AzTemplateS…
Script 4.8.0 Az.Storage {Add-AzRmStorageContainerLegalHold, Add-AzS…
Script 0.0.0.10 AzureAD.Standard.Preview {Add-AzureADApplicationOwner, Add-AzureADDe…
Script 0.9.3 AzurePSDrive
```

### Error output

```PowerShell
PS /home/mayank> Resolve-AzError
DEBUG: 8:31:21 AM - ResolveError begin processing with ParameterSet 'AnyErrorParameterSet'.
DEBUG: 8:31:21 AM - using account id 'MSI@50342'...
DEBUG: 8:31:21 AM - [ConfigManager] Got nothing from [DisplayBreakingChangeWarning], Module = [], Cmdlet = []. Returning default value [True].
WARNING: Upcoming breaking changes in the cmdlet 'Resolve-AzError' :
The `Resolve-Error` alias will be removed in a future release. Please change any scripts that use this alias to use `Resolve-AzError` instead.
Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other information on breaking changes in Azure PowerShell.

HistoryId: 7

Message : Exception type: ErrorResponseCommonV2Exception, Message:
Microsoft.Azure.Management.Monitor.Models.ErrorResponseCommonV2Exception: Operation returned an
invalid status code 'BadRequest'
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperations.CreateWithHttpMessagesAsy
nc(String resourceGroupName, String dataCollectionRuleName, DataCollectionRuleResource body,
Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperationsExtensions.CreateAsync(IDa
taCollectionRulesOperations operations, String resourceGroupName, String
dataCollectionRuleName, DataCollectionRuleResource body, CancellationToken cancellationToken)
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperationsExtensions.Create(IDataCol
lectionRulesOperations operations, String resourceGroupName, String dataCollectionRuleName,
DataCollectionRuleResource body)
at Microsoft.Azure.Commands.Insights.DataCollectionRules.SetAzureRmDataCollectionRuleCommand.
ReplaceDataCollectionRule()
at Microsoft.Azure.Commands.Insights.DataCollectionRules.SetAzureRmDataCollectionRuleCommand.
ProcessRecordInternal()
at Microsoft.Azure.Commands.Insights.MonitorCmdletBase.ExecuteCmdlet(), Code: Null, Status
code:Null, Reason phrase: Null
StackTrace : at Microsoft.Azure.Commands.Insights.MonitorCmdletBase.ExecuteCmdlet()
at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.<>c__3`1.b__3_0(T c)
at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAs
Job[T](T cmdlet, Action`1 executor)
at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAs
Job[T](T cmdlet)
at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception : System.Management.Automation.PSInvalidOperationException
InvocationInfo : {Set-AzDataCollectionRule}
Line : $dcr | Set-AzDataCollectionRule
Position : At line:1 char:8
+ $dcr | Set-AzDataCollectionRule
+ ~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId : 7

Message : Operation returned an invalid status code 'BadRequest'
StackTrace : at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperations.CreateWithHttpMessagesAsy
nc(String resourceGroupName, String dataCollectionRuleName, DataCollectionRuleResource body,
Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperationsExtensions.CreateAsync(IDa
taCollectionRulesOperations operations, String resourceGroupName, String
dataCollectionRuleName, DataCollectionRuleResource body, CancellationToken cancellationToken)
at Microsoft.Azure.Management.Monitor.DataCollectionRulesOperationsExtensions.Create(IDataCol
lectionRulesOperations operations, String resourceGroupName, String dataCollectionRuleName,
DataCollectionRuleResource body)
at Microsoft.Azure.Commands.Insights.DataCollectionRules.SetAzureRmDataCollectionRuleCommand.
ReplaceDataCollectionRule()
at Microsoft.Azure.Commands.Insights.DataCollectionRules.SetAzureRmDataCollectionRuleCommand.
ProcessRecordInternal()
at Microsoft.Azure.Commands.Insights.MonitorCmdletBase.ExecuteCmdlet()
Exception : Microsoft.Azure.Management.Monitor.Models.ErrorResponseCommonV2Exception
InvocationInfo : {Set-AzDataCollectionRule}
Line : $dcr | Set-AzDataCollectionRule
Position : At line:1 char:8
+ $dcr | Set-AzDataCollectionRule
+ ~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId : 7

HistoryId: 3

Message : Cannot validate argument on parameter 'RuleId'. The argument is null or empty. Provide an
argument that is not null or empty, and then try the command again.
StackTrace : at System.Management.Automation.ParameterBinderBase.BindParameter(CommandParameterInternal
parameter, CompiledCommandParameter parameterMetadata, ParameterBindingFlags flags)
at System.Management.Automation.CmdletParameterBinderController.BindParameter(CommandParamete
rInternal argument, MergedCompiledCommandParameter parameter, ParameterBindingFlags flags)
at System.Management.Automation.CmdletParameterBinderController.BindParameter(UInt32
parameterSets, CommandParameterInternal argument, MergedCompiledCommandParameter parameter,
ParameterBindingFlags flags)
at System.Management.Automation.CmdletParameterBinderController.BindNamedParameter(UInt32
parameterSets, CommandParameterInternal argument, MergedCompiledCommandParameter parameter)
at System.Management.Automation.ParameterBinderController.BindNamedParameters(UInt32
parameterSets, Collection`1 arguments)
at System.Management.Automation.CmdletParameterBinderController.BindCommandLineParametersNoVa
lidation(Collection`1 arguments)
at System.Management.Automation.CmdletParameterBinderController.BindCommandLineParameters(Col
lection`1 arguments)
at System.Management.Automation.CommandProcessor.BindCommandLineParameters()
at System.Management.Automation.CommandProcessor.Prepare(IDictionary
psDefaultParameterValues)
at System.Management.Automation.CommandProcessorBase.DoPrepare(IDictionary
psDefaultParameterValues)
at System.Management.Automation.Internal.PipelineProcessor.Start(Boolean incomingStream)
at
System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object
input)
--- End of stack trace from previous location ---
at
System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object
input)
at System.Management.Automation.PipelineOps.InvokePipeline(Object input, Boolean
ignoreInput, CommandParameterInternal[][] pipeElements, CommandBaseAst[] pipeElementAsts,
CommandRedirection[][] commandRedirections, FunctionContext funcContext)
at System.Management.Automation.Interpreter.ActionCallInstruction`6.Run(InterpretedFrame
frame)
at
System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame
frame)
Exception : System.Management.Automation.ParameterBindingValidationException
InvocationInfo : {Get-AzDataCollectionRule}
Line : Get-AzDataCollectionRule -RuleId $DCRid
Position : At line:1 char:34
+ Get-AzDataCollectionRule -RuleId $DCRid
+ ~~~~~~
HistoryId : 3

Message : The argument is null or empty. Provide an argument that is not null or empty, and then try the
command again.
StackTrace : at System.Management.Automation.ValidateNotNullOrEmptyAttribute.Validate(Object arguments,
EngineIntrinsics engineIntrinsics)
at System.Management.Automation.ParameterBinderBase.BindParameter(CommandParameterInternal
parameter, CompiledCommandParameter parameterMetadata, ParameterBindingFlags flags)
Exception : System.Management.Automation.ValidationMetadataException
InvocationInfo : {Get-AzDataCollectionRule}
Line : Get-AzDataCollectionRule -RuleId $DCRid
Position : At line:1 char:34
+ Get-AzDataCollectionRule -RuleId $DCRid
+ ~~~~~~
HistoryId : 3

The Azure PowerShell team is listening, please let us know how we are doing: https://aka.ms/azpssurvey?Q_CHL=ERROR.

DEBUG: 8:31:21 AM - [ConfigManager] Got [True] from [EnableDataCollection], Module = [], Cmdlet = [].
DEBUG: AzureQoSEvent: Module: Az.Accounts:2.9.1; CommandName: Resolve-AzError; PSVersion: 7.2.5; IsSuccess: True; Duration: 00:00:00.0213725
DEBUG: Finish sending metric.
DEBUG: 8:31:22 AM - ResolveError end processing.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.