elastic / elastic/beats

Processor add_cloud_metadata adds incorrect values to cloud metadata fields

Open
#47,291 1 comment 0 reactions 0 assignees View on GitHub
Team:obs-ds-hosted-services
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

When running elastic stack on self-managed on-prem cluster (100% of their Windows hosts are physical, on-prem machines), a large volume of the ingested logs contain `cloud.*` fields. This is unexpected since none of these hosts are cloud VMs.
For example:
```
cloud.provider: openstack
cloud.service.name: Nova
```

This issue is caused by the request that checks if the agent is running on AWS (169.254.169.254) being intercepted by a possible Fortinet firewall. Instead of timing out as it should if it is not an AWS host, it returns an HTML redirect, which prevents the agent from running a proper check. Because of this incorrect response, the agent mistakenly sets the host's cloud provider to `openstack`.

```
window.location=\"https://169.254.169.254:80/fgtauth?000110ab149ec7aa\";
```

Instead of populating the cloud fields with the response of a firewall, `add_cloud_metadata` processor should verify the response from the endpoints first.

Contributor guide

Open the contributing guide

Research direction

Start at the add_cloud_metadata processor and its AWS metadata endpoint check, then inspect how intercepted HTML responses are handled. Reproduce the Fortinet-style redirect response and verify that invalid endpoint responses no longer populate cloud fields; add or update coverage for this behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cloud, observability-sre
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.