fluent / fluent/fluent-bit

Filters – AWS Metadata – Support Additional IMDS Instance Metadata Categories

Open
#9,640 6 comments 1 reaction 0 assignees View on GitHub
exempt-stale
Dominant language
C
Stars
8.1k
Forks
2k
Avg merge
4d 16h
Merged PRs (30d)
58

Description

### Is your feature request related to a problem? Please describe.

The [AWS Metadata filter](https://docs.fluentbit.io/manual/pipeline/filters/aws-metadata) currently supports a small subset of [IMDS instance metadata categories](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-categories).

### Describe the solution you'd like

Support additional IMDS instance metadata categories. In particular:

- __Services__
- __AWS Partition (`services/partition`)__
- __AWS Region Domain (`services/domain`)__
- __Placement__
- __AWS Region (`placement/region`)__
- In multi-region deployments with global observability aggregation, it's useful to know an instance's region to identify region outages (e.g. `us-east-1` is down but `us-west-2` is up).
- __[AWS Availability Zone ID](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#az-ids) (`placement/availability-zone-id`)__
- Across AWS accounts, the same availability zone name (e.g. `us-east-1a`, which the filter supports today) maps to different underlying availability zone IDs (e.g. `use1-az1`, `use1-az2`).
- In multi-account deployments, it's useful to know an instance's availability zone ID to identify availability zone outages (e.g. `use1-az1` is down but `use1-az2` is up).
- __Placement Group Name (`placement/group-name`)__
- __Placement Group Partition Number (`placement/partition-number`)__
- __Dedicated Host ID (`placement/host-id`)__
- Useful for identifying bad dedicated hosts (e.g. EC2 Mac instances which require dedicated hosts due to Apple's EULA).
- __IP Addresses (Public/Private IPv4/6)__
- The filter currently supports only the private IPv4 address.
- __AWS EC2 Auto Scaling__
- __Target Lifecycle State (`autoscaling/target-lifecycle-state`)__
- Useful for correlating [EC2 Auto Scaling Lifecycle Hook](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html) events with on-instance events (e.g. graceful shutdown procedures).

### Describe alternatives you've considered

N/A

### Additional context

N/A

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.