aws-for-fluent-bit auto_create_group option is always disabled for cloudwatchlogs plugin in fluentbit helm chart
- Dominant language
- Mustache
- Stars
- 1.3k
- Forks
- 1.1k
- Avg merge
- 22m
- Merged PRs (30d)
- 5
Description
**Describe the bug**
auto_create_group option is disabled, the pod reports this error below repeteadly, if I use cloudwatch intead of cloudwatchlogs plugin within the fluentbit helm values the log group is created, but that plugin has worst performance and has not templating options.
2024/04/23 13:37:50] [ warn] [engine] failed to flush chunk '1-1713879469.228413452.flb', retry in 10 seconds: task_id=7, input=tail.0 > output=cloudwatch_logs.1 (out_id=1)
[2024/04/23 13:37:50] [error] [output:cloudwatch_logs:cloudwatch_logs.1] Log Group /aws/eks/mycluster/aws-fluentbit-logs not found and auto_create_group disabled.
[2024/04/23 13:37:50] [error] [output:cloudwatch_logs:cloudwatch_logs.1] Failed to send events
[2024/04/23 13:37:50] [ warn] [engine] failed to flush chunk '1-1713879469.551363209.flb', retry in 10 seconds: task_id=8, input=tail.0 > output=cloudwatch_logs.1 (out_id=1)
[2024/04/23 13:37:50] [ warn] [engine] chunk '1-1713879459.223247680.flb' cannot be retried: task_id=3, input=tail.0 > output=cloudwatch_logs.1
**Steps to reproduce**
Just enable cloudwatchlogs, i tested different image versions so i dont think the issue is in the image.
image:
repository: public.ecr.aws/aws-observability/aws-for-fluent-bit
tag: "2.28.5"
pullPolicy: IfNotPresent
…
cloudWatchLogs:
autoCreateGroup: true
enabled: true
match: "*"
region: "us-east-1"
logGroupTemplate: "/aws/eks/mycluster/$kubernetes['host'].$kubernetes['namespace_name']"
logStreamTemplate: "$kubernetes['pod_name'].$kubernetes['container_name']"
logRetentionDays: 14
**Expected outcome**
log group should be created , only old plugin cloudwatch does but not cloudwatchlogs
APPLICATION VERSION
2.31.12.20231011
* Chart name: aws-for-fluent-bit
* Chart version: 0.1.32
* Using EKS (yes/no), if so version? yes 1.29
Contributor guide
Research direction
Start with the aws-for-fluent-bit chart values shown in the issue and inspect the rendered configuration for the cloudwatchlogs plugin. Compare the autoCreateGroup setting with the emitted auto_create_group option, then reproduce with the provided values; done means the option is enabled and the missing log group is created.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100