[filter_kubernetes] enhancement: provide mechanism to exclude containers from fluent bit via annotations
- Dominant language
- C
- Stars
- 8.1k
- Forks
- 2k
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 71
Description
**Problem Statement**
In the current implementation the most common way to get container logs parsed by fluentbit in a kubernetes cluster is to have a filter applied to containers using a log message like the one detailed here:
```
input-kubernetes.conf: |
[INPUT]
Name tail
Tag kube.*
Path /var/log/containers/*.log
Parser docker
DB /var/log/flb_kube.db
```
This represents a bit of a problem with dev containers sometimes polluting our log platform with unwanted logs. We have a select few services we want to include, but the default rule is to exclude.
**Describe the solution you'd like**
A simple way to include logs in the opposite of how #555 was implemented where you can annotate containers with a `fluentbit.io/include: "true"` and the fluentbit daemonset will only pick up these logs.
**Describe alternatives you've considered**
I've updated my Path in the above config to `/var/log/containers/*deployment*.log` and ensured all my deployments I want to aggregate logs for have `deployment` in the name.
**Additional context**
I have a Kubernetes cluster set up with kops 1.10 and used https://github.com/fluent/fluent-bit-kubernetes-logging to set up fluentbit which then forwards to a fluentd service running the logz.io plugin.
Contributor guide
Assessment
This issue has not been assessed yet.