aws / aws/containers-roadmap

[EKS] [Fargate]: have fluentbit kubernetes filter honor requests for namespace metadata

Open
#2,699 0 comments 0 reactions 0 assignees View on GitHub
EKS Fargate Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
What do you want us to build?
> Have aws-for-fluent-bit attach `kubernetes_namespace` object to log records when kubernetes filter contains `Namespace_Labels` or `Namespace_Annotations` options set.

**Which service(s) is this request for?**
> Fargate on EKS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
> Kubernetes pod metadata allows us to build a fluentbit configuration that drops log records for pods with a target annotation or label. I am working on a use case that requires that we toggle logging on and off for all workload resources of a target namespace.

> With access to metadata at the namespace level, we could add filters in the fluentbit configuration to drop records when target label/annotation is found at the namespace level which would greatly simplify the toggle of logging for a namespace (if we rely on pod metadata only, toggle would require editing the drop label/annotation for all pods in the namespace which is expensive and error prone)

**Are you currently working around this issue?**
How are you currently solving this problem?
> I am currently studying alternatives to namespace metadata in order to make this design work well but so far I did not manage to find a work around

**Additional context**
Anything else we should know?

**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

I have enabled process logs for the fluentbit running on Fargate and I see its fluentbit version is 4.0.1. Namespace metadata is a feature introduced in fluentbit 3.0 so I would expect this to work
```

{ "log": "Fluent Bit v4.0.1" }
--
{ "log": "[2025/10/21 14:38:43] [ info] [fluent bit] version=4.0.1, commit=, pid=" }

```

Here is a minimum example of what a configmap to reproduce this looks like
```

kind: Namespace
apiVersion: v1
metadata:
name: aws-observability
labels:
aws-observability: enabled
---
apiVersion: v1
kind: ConfigMap
metadata:
name: aws-logging
namespace: aws-observability
data:
flb_log_cw: 'true' # <- make sure pod execution role has cloudwatch log permissions
filters.conf: |
[FILTER]
Name kubernetes
Match kube.*
Merge_Log On
Kube_Meta_Cache_TTL 30s
Namespace_Labels On # <- should attach namespace labels map to "kubernetes_namespace" object
Namespace_Annotations On # <- should attach namespace annotations map to "kubernetes_namespace" object
Kube_Meta_Namespace_Cache_TTL 30s
output.conf: |
[OUTPUT]
Name cloudwatch
Match kube.*
region
log_group_name fluent-bit-cloudwatch
log_stream_prefix from-fluent-bit-
auto_create_group true

```

log output in cloudwatch destination has no `kubernetes_namespace` object and looks similar to the following:
```

{
"kubernetes": {
"annotations": {
"CapacityProvisioned": "0.25vCPU 0.5GB",
"Logging": "LoggingEnabled”,
...
},
"container_image": “",
"container_name": “",
"host": "fargate-ip-..compute.internal",
"labels": {
"eks.amazonaws.com/fargate-profile": “",
"pod-template-hash": “",
...
},
"namespace_name": “",
"pod_id": “",
"pod_name": “-668cc5567b-q5w5z"
},
"log": “"
}

```

Especially because we seem to be using the latest fluentbit major as the base for this aws fork, I think it would be great if we supported this namespace feature. I am sure it is useful for other use cases out there.

Contributor guide

Open the contributing guide

Research direction

The issue names no repository files, tests, or implementation entry points. Start by locating the aws-for-fluent-bit Kubernetes filter integration and its handling of Namespace_Labels and Namespace_Annotations, then determine how Fargate supplies namespace metadata. Done means configurations using those options attach a kubernetes_namespace object with the requested metadata and have coverage for the provided reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
cloud, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.