fluent / fluent/fluent-operator
Make namespaced rewrite_tag rule key configurable
- Dominant language
- Go
- Stars
- 682
- Forks
- 328
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 13
Description
### Is your feature request related to a problem? Yes, read below.
Hi everyone,
Using the namespaced CRs to deploy specific FluentBit configuration deploys the dynamic Filter similar to below.
```yaml
[Filter]
Name rewrite_tag
Match kube.*
Rule $kubernetes['namespace_name'] ^(test)$ 5236c06addf811b0eaf2fabbd31c89f2.$TAG false
```
This doesn't work in case you have syslog configured as an output, and need to list the nested objects. Check out [this issue](https://github.com/fluent/fluent-bit/issues/5467) for more information.
Looking at the code, we found the below line to be hard-coded.
```go
buf.WriteString(fmt.Sprintf(" Rule $kubernetes['namespace_name'] ^(%s)$ %x.$TAG false\n", cfg.Namespace, md5.Sum([]byte(cfg.Namespace))))
```
[Source](https://github.com/fluent/fluent-operator/blob/9d10cd98c5dd824305fece48b3fc9ca7cfd552de/controllers/fluentbitconfig_controller.go#L312)
This setting causes the `rewrite_tag` filter not to work properly.
### Describe the solution you'd like
Some possible solution would be somewhat similar to the one used in [this PR](https://github.com/fluent/fluent-operator/pull/1085).
For example:
```yaml
apiVersion: fluentbit.fluent.io/v1alpha2
kind: FluentBitConfig
metadata:
name: fluent-bit-config
labels:
app.kubernetes.io/name: fluent-bit
spec:
filterSelector:
matchLabels:
fluentbit.fluent.io/enabled: "true"
outputSelector:
matchLabels:
fluentbit.fluent.io/enabled: "true"
parserSelector:
matchLabels:
fluentbit.fluent.io/enabled: "true"
service:
emitterRuleKey: "kubernetes_namespace_name"
```
Would generate to the below filter.
```yaml
[Filter]
Name rewrite_tag
Match kube.*
Rule $kubernetes_namespace_name ^(test)$ 5236c06addf811b0eaf2fabbd31c89f2.$TAG false
Emitter_Name test
```
### Additional context
Some useful links:
- https://docs.fluentbit.io/manual/pipeline/filters/rewrite-tag#key
- https://github.com/fluent/fluent-operator/pull/1085
- https://github.com/fluent/fluent-bit/issues/5467
- https://github.com/fluent/fluent-operator/blob/release-2.7/controllers/fluentbitconfig_controller.go#L312
Thanks,
Marjan
Contributor guide
Research direction
Start in controllers/fluentbitconfig_controller.go around line 312 and compare the service setting with the approach in PR #1085. Read Fluent Bit's rewrite_tag key documentation and issue #5467 first. Done means the namespaced rewrite_tag Rule key can be configured through the FluentBitConfig service setting and generated output uses that key.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100