fluent / fluent/fluent-operator

Make namespaced rewrite_tag rule key configurable

Open
#1,096 0 comments 8 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
682
Forks
328
Avg merge
2d 23h
Merged PRs (30d)
13

Description

### Is your feature request related to a problem? Yes, read below.

Hi everyone,

Using the namespaced CRs to deploy specific FluentBit configuration deploys the dynamic Filter similar to below.

```yaml
[Filter]
Name rewrite_tag
Match kube.*
Rule $kubernetes['namespace_name'] ^(test)$ 5236c06addf811b0eaf2fabbd31c89f2.$TAG false
```

This doesn't work in case you have syslog configured as an output, and need to list the nested objects. Check out [this issue](https://github.com/fluent/fluent-bit/issues/5467) for more information.

Looking at the code, we found the below line to be hard-coded.

```go
buf.WriteString(fmt.Sprintf(" Rule $kubernetes['namespace_name'] ^(%s)$ %x.$TAG false\n", cfg.Namespace, md5.Sum([]byte(cfg.Namespace))))
```
[Source](https://github.com/fluent/fluent-operator/blob/9d10cd98c5dd824305fece48b3fc9ca7cfd552de/controllers/fluentbitconfig_controller.go#L312)

This setting causes the `rewrite_tag` filter not to work properly.

### Describe the solution you'd like

Some possible solution would be somewhat similar to the one used in [this PR](https://github.com/fluent/fluent-operator/pull/1085).

For example:

```yaml
apiVersion: fluentbit.fluent.io/v1alpha2
kind: FluentBitConfig
metadata:
name: fluent-bit-config
labels:
app.kubernetes.io/name: fluent-bit
spec:
filterSelector:
matchLabels:
fluentbit.fluent.io/enabled: "true"
outputSelector:
matchLabels:
fluentbit.fluent.io/enabled: "true"
parserSelector:
matchLabels:
fluentbit.fluent.io/enabled: "true"
service:
emitterRuleKey: "kubernetes_namespace_name"
```

Would generate to the below filter.

```yaml
[Filter]
Name rewrite_tag
Match kube.*
Rule $kubernetes_namespace_name ^(test)$ 5236c06addf811b0eaf2fabbd31c89f2.$TAG false
Emitter_Name test
```

### Additional context

Some useful links:

- https://docs.fluentbit.io/manual/pipeline/filters/rewrite-tag#key
- https://github.com/fluent/fluent-operator/pull/1085
- https://github.com/fluent/fluent-bit/issues/5467
- https://github.com/fluent/fluent-operator/blob/release-2.7/controllers/fluentbitconfig_controller.go#L312

Thanks,
Marjan

Contributor guide

Open the contributing guide

Research direction

Start in controllers/fluentbitconfig_controller.go around line 312 and compare the service setting with the approach in PR #1085. Read Fluent Bit's rewrite_tag key documentation and issue #5467 first. Done means the namespaced rewrite_tag Rule key can be configured through the FluentBitConfig service setting and generated output uses that key.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.