Azure / Azure/azure-powershell

Invoke-AzOperationalInsightsQuery empty results set

Open
#22,844 4 comments 0 reactions 0 assignees View on GitHub
bug customer-reported
Dominant language
C#
Stars
4.8k
Forks
4.3k
Avg merge
2d 17h
Merged PRs (30d)
51

Description

### Description

Invoke-AzOperationalInsightsQuery was providing results until late afternoon yesterday. Using the same query to send and now get empty results. Have tried simpler queries and they fail as well. Also have taken the query created in powershell and confirmed that the query is properly formatted and works in log analytics in the portal. When running the query in the portal instead of powershell, results are returned.

Debug output attached with just one of a dozen queries attempted that used to work and no longer work.

### Issue script & Debug output

```PowerShell
DEBUG: 9:15:27 AM - [ConfigManager] Got nothing from [DisplayRegionIdentified],
Module = [], Cmdlet = []. Returning default value [True].
DEBUG: 9:15:27 AM - [ConfigManager] Got nothing from [CheckForUpgrade], Module =
[], Cmdlet = []. Returning default value [False].
DEBUG: AzureQoSEvent: Module: Az.OperationalInsights:3.2.0; CommandName: Get-Az
OperationalInsightsWorkspace; PSVersion: 5.1.19041.3031; IsSuccess: True; Durati
on: 00:00:00.8094235
DEBUG: 9:15:27 AM - [ConfigManager] Got nothing from [EnableDataCollection], Mod
ule = [], Cmdlet = []. Returning default value [True].
DEBUG: 9:15:27 AM - GetAzureOperationalInsightsWorkspaceCommand end processing.
DEBUG: 9:15:27 AM - InvokeOperationalInsightsQuery begin processing with Paramet
erSet 'ByWorkspaceId'.
DEBUG: 9:15:27 AM - using account id 'xxxxxx@xxxx.com'...
DEBUG: 9:15:27 AM - [ConfigManager] Got nothing from [DisplayBreakingChangeWarni
ng], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: [Common.Authentication]: Authenticating using Account: 'xxxxx@xxxxx.c
om', environment: 'AzureCloud', tenant: '7389d8c0-3607-465c-a69f-7d4426502912'
DEBUG: 9:15:27 AM - [ConfigManager] Got nothing from [EnableLoginByWam], Module
= [], Cmdlet = []. Returning default value [False].
DEBUG: 9:15:27 AM - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'7389d8c0-3607-465c-a69f-7d4426502912', Scopes:'https://api.loganalytics.io/.default', AuthorityHost:
'https://login.microsoftonline.com/', UserId:'xxxxxxx@xxxxxx.com'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://api.loganalytics.io/.default ] ParentRequestId:
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - bc6faa47-8dde-4ba3-bd51-b852c31da108] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - bc6faa47-8dde-4ba3-bd51-b852c31da108] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - bc6faa47-8dde-4ba3-bd51-b852c31da108] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - bc6faa47-8dde-4ba3-bd51-b852c31da108] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - bc6faa47-8dde-4ba3-bd51-b852c31da108] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z] Found 1 cache accounts and 0 broker accounts
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z] Returning 1 accounts
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] MSAL MSAL.Desktop with assembly version '4.49.1.0'. CorrelationId(
1b9a44d9-314a-42e9-91cb-49a655765785)
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] === AcquireTokenSilent Parameters ===
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] LoginHint provided: False
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] Account provided: True
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] ForceRefresh: False
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785]
=== Request Data ===
Authority Provided? - True
Scopes - https://api.loganalytics.io/.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - 1b9a44d9-314a-42e9-91cb-49a655765785
UserAssertion set: False
LongRunningOboCacheKey set: False
Region configured:

DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] === Token Acquisition (SilentRequest) started:
Scopes: https://api.loganalytics.io/.default
Authority Host: login.microsoftonline.com
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] Access token has expired or about to expire. [Current time (09/21/
2023 15:15:27) - Expiration Time (09/21/2023 15:08:37 +00:00) - Extended Expiration Time (09/21/2023 15:08:37 +00:00)]
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [FindRefreshTokenAsync] Refresh token found in the cache? - True
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [Instance Discovery] Instance discovery is enabled and will be per
formed
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:27Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [Region discovery] Not using a regional authority.
DEBUG: Request [8fdf6bb1-4663-4d0d-8fc1-fce6bb6abbde] POST https://login.microsoftonline.com/7389d8c0-3607-465c-a69f-7d4426502912/oauth2/v2.0/token
x-client-SKU:REDACTED
x-client-Ver:REDACTED
x-client-CPU:REDACTED
x-client-OS:REDACTED
x-anchormailbox:REDACTED
x-client-current-telemetry:REDACTED
x-client-last-telemetry:REDACTED
x-ms-lib-capability:REDACTED
client-request-id:REDACTED
return-client-request-id:REDACTED
x-app-name:REDACTED
x-app-ver:REDACTED
Content-Type:application/x-www-form-urlencoded
x-ms-client-request-id:8fdf6bb1-4663-4d0d-8fc1-fce6bb6abbde
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Identity/1.6.1 (.NET Framework 4.8.4645.0; Microsoft Windows 10.0.19045 )
client assembly: Azure.Identity
DEBUG: Response [8fdf6bb1-4663-4d0d-8fc1-fce6bb6abbde] 200 OK (00.4s)
Pragma:no-cache
Strict-Transport-Security:REDACTED
X-Content-Type-Options:REDACTED
client-request-id:REDACTED
x-ms-request-id:7c78f01c-fcff-4f99-a740-12e2ea468500
x-ms-ests-server:REDACTED
x-ms-clitelem:REDACTED
X-XSS-Protection:REDACTED
Cache-Control:no-store, no-cache
Content-Type:application/json; charset=utf-8
Expires:-1
P3P:REDACTED
Set-Cookie:REDACTED
Date:Thu, 21 Sep 2023 15:15:28 GMT
Content-Length:10570

DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] Checking client info returned from the server..
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] Saving token response to cache..
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [Region discovery] Not using a regional authority.
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [SaveTokenResponseAsync] Saving AT in cache and removing overlappi
ng ATs...
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] Looking for scopes for the authority in the cache which intersect
with https://api.loganalytics.io/.default
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] Intersecting scope entries count - 1
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] Matching entries after filtering by user - 1
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [SaveTokenResponseAsync] Saving Id Token and Account in cache ...
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] [SaveTokenResponseAsync] Saving RT in cache...
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] IsLegacyAdalCacheEnabled: yes
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] Not writing FRT in ADAL legacy cache.
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785]
=== Token Acquisition finished successfully:
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] AT expiration time: 9/21/2023 4:23:16 PM +00:00, scopes: https://
api.loganalytics.io/user_impersonation https://api.loganalytics.io/.default. source: IdentityProvider
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows 10 Enterprise [2023-09-21 15:15:28Z - 1b9a44d9-314a-42e9-91cb-49a655765785] Fetched access token from host login.microsoftonline.com.
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://api.loganalytics.io/.default ] ParentRequestId: ExpiresOn: 2023-09-21T16:23:16.1496957+00:00
DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: '7389d8c0-3607-465c-a69f-7d4426502912', UserId: 'xxxxxxx@xxxxxxx.com'
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
POST

Absolute Uri:
https://api.loganalytics.io/v1/workspaces/16a22acc-8d84-4e3f-bdc3-b07e823ce194/query

Headers:
x-ms-client-request-id : 704ffd19-5c2b-4efd-9698-ecacd2843230

Body:
{
"query": "let PercentSpace = 50; //enter the threshold for the disk space percentage\r\n Perf\r\n | where ObjectName == 'LogicalDisk' and CounterName == '% Free Space'\r\n | where TimeGenerate
d > ago(30d)\r\n | summarize FreeSpace = min(CounterValue) by Computer, InstanceName\r\n // | where InstanceName == 'C:' or InstanceName == 'D:' or InstanceName == 'E:'\r\n | where InstanceName
has ':' //has is recommended over contains due to cost and cpu\r\n // look for the colon in the drive letter\r\n | where FreeSpace > PercentSpace //and Computer has 'servername'\r\n | proje
ct Computer, FreeSpace, Disk = InstanceName //, _ResourceId, _SubscriptionId, SourceSystem, StandardDeviation\r\n // set to < in normal circumstances\r\n | sort by FreeSpace asc\r\n "
}

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Connection : keep-alive
X-Content-Type-Options : nosniff
Access-Control-Allow-Origin : *
Access-Control-Expose-Headers : Retry-After,Age,WWW-Authenticate,x-resource-identities,x-ms-status-location
Vary : Accept-Encoding
Strict-Transport-Security : max-age=15724800; includeSubDomains
Date : Thu, 21 Sep 2023 15:15:30 GMT
Via : 1.1 draft-oms-7f55889fb9-8nkf6

Body:
{
"tables": [
{
"name": "PrimaryResult",
"columns": [
{
"name": "Computer",
"type": "string"
},
{
"name": "FreeSpace",
"type": "real"
},
{
"name": "Disk",
"type": "string"
}
],
"rows": []
}
],
"render": {
"visualization": null,
"title": null,
"accumulate": false,
"isQuerySorted": true,
"kind": null,
"legend": null,
"series": null,
"yMin": "NaN",
"yMax": "NaN",
"xAxis": null,
"xColumn": null,
"xTitle": null,
"yAxis": null,
"yColumns": null,
"ySplit": null,
"yTitle": null,
"anomalyColumns": null
},
"statistics": {
"query": {
"executionTime": 0,
"resourceUsage": {
"cache": {
"memory": {
"hits": 0,
"misses": 0,
"total": 0
},
"disk": {
"hits": 0,
"misses": 0,
"total": 0
},
"shards": {
"hot": {
"hitbytes": 0,
"missbytes": 0,
"retrievebytes": 0
},
"cold": {
"hitbytes": 0,
"missbytes": 0,
"retrievebytes": 0
},
"bypassbytes": 0
}
},
"cpu": {
"user": "00:00:00",
"kernel": "00:00:00",
"totalCpu": "00:00:00"
},
"memory": {
"peakPerNode": 524384
},
"network": {
"interClusterTotalBytes": 512,
"crossClusterTotalBytes": 0
}
},
"inputDatasetStatistics": {
"extents": {
"total": 0,
"scanned": 0,
"scannedMinDatetime": "0001-01-01T00:00:00Z",
"scannedMaxDatetime": "0001-01-01T00:00:00Z"
},
"rows": {
"total": 0,
"scanned": 0
},
"rowstores": {
"scannedRows": 0,
"scannedValuesSize": 0
},
"shards": {
"queriesGeneric": 0,
"queriesSpecialized": 0
}
},
"datasetStatistics": [
{
"tableRowCount": 0,
"tableSize": 0
}
],
"crossClusterResourceUsage": {}
}
}
}

DEBUG: 9:15:31 AM - [ConfigManager] Got nothing from [DisplayRegionIdentified], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: 9:15:31 AM - [ConfigManager] Got nothing from [CheckForUpgrade], Module = [], Cmdlet = []. Returning default value [False].
DEBUG: AzureQoSEvent: Module: Az.OperationalInsights:3.2.0; CommandName: Invoke-AzOperationalInsightsQuery; PSVersion: 5.1.19041.3031; IsSuccess: True; Duration: 00:00:03.2896861
DEBUG: 9:15:31 AM - [ConfigManager] Got nothing from [EnableDataCollection], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: 9:15:31 AM - InvokeOperationalInsightsQuery end processing.
16a22acc-8d84-4e3f-bdc3-b07e823ce194
let PercentSpace = 50; //enter the threshold for the disk space percentage
Perf
| where ObjectName == 'LogicalDisk' and CounterName == '% Free Space'
| where TimeGenerated > ago(30d)
| summarize FreeSpace = min(CounterValue) by Computer, InstanceName
// | where InstanceName == 'C:' or InstanceName == 'D:' or InstanceName == 'E:'
| where InstanceName has ':' //has is recommended over contains due to cost and cpu
// look for the colon in the drive letter
| where FreeSpace > PercentSpace //and Computer has 'servername'
| project Computer, FreeSpace, Disk = InstanceName //, _ResourceId, _SubscriptionId, SourceSystem, StandardDeviation
// set to < in normal circumstances
| sort by FreeSpace asc

Results : {}
Render : {[visualization, ], [title, ], [accumulate, false], [isQuerySorted, true]...}
Statistics : {[query, {
"executionTime": 0,
"resourceUsage": {
"cache": {
"memory": {
"hits": 0,
"misses": 0,
"total": 0
},
"disk": {
"hits": 0,
"misses": 0,
"total": 0
},
"shards": {
"hot": {
"hitbytes": 0,
"missbytes": 0,
"retrievebytes": 0
},
"cold": {
"hitbytes": 0,
"missbytes": 0,
"retrievebytes": 0
},
"bypassbytes": 0
}
},
"cpu": {
"user": "00:00:00",
"kernel": "00:00:00",
"totalCpu": "00:00:00"
},
"memory": {
"peakPerNode": 524384
},
"network": {
"interClusterTotalBytes": 512,
"crossClusterTotalBytes": 0
}
},
"inputDatasetStatistics": {
"extents": {
"total": 0,
"scanned": 0,
"scannedMinDatetime": "0001-01-01T00:00:00Z",
"scannedMaxDatetime": "0001-01-01T00:00:00Z"
},
"rows": {
"total": 0,
"scanned": 0
},
"rowstores": {
"scannedRows": 0,
"scannedValuesSize": 0
},
"shards": {
"queriesGeneric": 0,
"queriesSpecialized": 0
}
},
"datasetStatistics": [
{
"tableRowCount": 0,
"tableSize": 0
}
],
"crossClusterResourceUsage": {}
}]}
Error :

Key : visualization
Value :

Key : title
Value :

Key : accumulate
Value : false

Key : isQuerySorted
Value : true

Key : kind
Value :

Key : legend
Value :

Key : series
Value :

Key : yMin
Value : NaN

Key : yMax
Value : NaN

Key : xAxis
Value :

Key : xColumn
Value :

Key : xTitle
Value :

Key : yAxis
Value :

Key : yColumns
Value :

Key : ySplit
Value :

Key : yTitle
Value :

Key : anomalyColumns
Value :

Key : query
Value : {executionTime, resourceUsage, inputDatasetStatistics, datasetStatistics...}
```

### Environment data

```PowerShell
PSVersion: 5.1.19041.3031
```

### Module versions

```PowerShell
ModuleType Version Name
---------- ------- ----
Script 2.13.0 Az.Accounts
Script 3.2.0 Az.OperationalInsights
```

### Error output

```PowerShell
DEBUG: 9:35:25 AM - [ConfigManager] Got nothing from [DisplayRegionIdentified], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: 9:35:25 AM - [ConfigManager] Got nothing from [CheckForUpgrade], Module = [], Cmdlet = []. Returning default value [False].
DEBUG: AzureQoSEvent: Module: Az.Accounts:2.13.0; CommandName: Resolve-AzError; PSVersion: 5.1.19041.3031; IsSuccess: True; Duration: 00:00:00.0512895
DEBUG: 9:35:25 AM - [ConfigManager] Got nothing from [EnableDataCollection], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: 9:35:25 AM - ResolveError end processing.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.