elastic / elastic/beats

[Azure monitor] ignore_unsupported does not skip HTTP 400 for unsupported metric namespaces

Open
#52,999 1 comment 0 reactions 0 assignees View on GitHub
bug input:azure-monitor Team:obs-ds-hosted-services
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

## Summary

`ignore_unsupported: true` should skip a resource when its namespace is not a supported Azure Monitor platform metric namespace. Today a HTTP 400 from the `metricDefinitions` API is still fatal. The `azure.monitor` metricset returns the error. The Elastic Agent unit changes from `HEALTHY` to `DEGRADED`. Collection stops for every resource in that unit, including types that are valid.

This is the Beats defect behind [elastic/integrations#20848](https://github.com/elastic/integrations/issues/20848).

## Version

- Elastic Stack 9.5.2
- Elastic Agent 9.4.4
- Azure Resource Metrics integration 1.14.0
- Current `main` still has the same path in `x-pack/metricbeat/module/azure`

## Operating System

Linux (AKS). The failure is in the Azure Monitor API client, not in the OS.

## Expected behaviour

When `ignore_unsupported` is true and Azure returns HTTP 400 because the namespace is not a supported platform metric namespace:

1. Metricbeat logs an info or warning line.
2. Metricbeat skips that resource.
3. Metricbeat continues collection for the other resources in the same cycle.
4. The unit stays `HEALTHY`.

That is the documented purpose of `ignore_unsupported` on `MetricConfig`.

## Actual behaviour

`GetMetricDefinitionsWithRetry` returns the HTTP 400. `mapMetrics` returns that error before it reads `IgnoreUnsupported`. `InitResources` then fails the whole refresh. The Agent unit becomes `DEGRADED`.

`ignore_unsupported` only applies after a successful definitions call that returns an empty list. It does not apply to a 400 from Azure.

Example error:

```
Error fetching data for metricset azure.monitor: GET https://management.azure.com/subscriptions//resourceGroups//providers/Microsoft.Insights/activityLogAlerts//providers/Microsoft.Insights/metricDefinitions
RESPONSE 400: 400 Bad Request
ERROR CODE: BadRequest
{
"code": "BadRequest",
"message": "Microsoft.Insights/activityLogAlerts is not a supported platform metric namespace, ..."
}
```

Unit state: `HEALTHY` to `DEGRADED`.

## Affected code

### 1. HTTP 400 is fatal before `ignore_unsupported` is checked

`GetMetricDefinitionsWithRetry` returns on error before `IgnoreUnsupported` is checked:

- Sequential:

https://github.com/elastic/beats/blob/99112f7e4dd543d01405d0461721a92d8960f8b9/x-pack/metricbeat/module/azure/monitor/client_helper.go#L39-L42

- Concurrent:

https://github.com/elastic/beats/blob/99112f7e4dd543d01405d0461721a92d8960f8b9/x-pack/metricbeat/module/azure/monitor/client_helper_concurrent.go#L49-L52

`ignore_unsupported` is only used later, when the call succeeds and the list is empty:

- Sequential:

https://github.com/elastic/beats/blob/99112f7e4dd543d01405d0461721a92d8960f8b9/x-pack/metricbeat/module/azure/monitor/client_helper.go#L46-L53

- Concurrent:

https://github.com/elastic/beats/blob/99112f7e4dd543d01405d0461721a92d8960f8b9/x-pack/metricbeat/module/azure/monitor/client_helper_concurrent.go#L56-L61

### 2. Non-429 errors are always returned

`GetMetricDefinitionsWithRetry` and `sleepIfPossible` retry HTTP 429 only. HTTP 400 is returned as an error.

https://github.com/elastic/beats/blob/99112f7e4dd543d01405d0461721a92d8960f8b9/x-pack/metricbeat/module/azure/monitor_service.go#L227-L228

https://github.com/elastic/beats/blob/99112f7e4dd543d01405d0461721a92d8960f8b9/x-pack/metricbeat/module/azure/monitor_service.go#L265-L268

### 3. One mapping error fails the whole unit

`InitResources` returns on the first `mapMetrics` error. Remaining resource configs are not processed.

https://github.com/elastic/beats/blob/99112f7e4dd543d01405d0461721a92d8960f8b9/x-pack/metricbeat/module/azure/client.go#L119-L121

Contributor guide

Open the contributing guide

Research direction

Start with GetMetricDefinitionsWithRetry in x-pack/metricbeat/module/azure/monitor/client_helper.go and client_helper_concurrent.go, then trace mapMetrics and InitResources in client.go. Check the retry and error paths in monitor_service.go; done means an unsupported-namespace HTTP 400 is skipped when ignore_unsupported is true, other resources continue, and the unit remains HEALTHY.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, go
Domain
cloud, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.