aws / aws/containers-roadmap

[EKS][request]: Fargate Logging add support for `cri` Parser

Open
#1,366 0 comments 2 reactions 0 assignees View on GitHub
EKS Fargate Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**

I would like to see "cri" parser support added for [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html) as each/every log entry written appears to be "cri" format.

**Which service(s) is this request for?**

EKS Fargate Logging

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**

The problem I'm trying to solve is log parsing using a supported parser (as opposed to implementing/configuring the parser myself). The larger problem I'm trying to solve is the use of EKS Fargate Logging use across all AWS customers, as each/every customer would (if they wanted properly parsed logs to CloudWatch) need to configure a log parser themselves.

**Are you currently working around this issue?**

I am currently adding the log parser myself, by using the following:

```
---
kind: ConfigMap
apiVersion: v1
metadata:
name: aws-logging
namespace: aws-observability
data:
parsers.conf: |
[PARSER]
Name fluentbit_for_eks_fargate
Format regex
Regex ^(?

**Additional context**

I would prefer the implementation shown below:

```
---
kind: ConfigMap
apiVersion: v1
metadata:
name: aws-logging
namespace: aws-observability
data:
filters.conf: |
[FILTER]
Name cri
Match from-fluent-bit-kube.*
Key_Name log
Parser cri
output.conf: |
[OUTPUT]
Name cloudwatch
Match *
region us-west-2
log_group_name fluent-bit-cloudwatch
log_stream_prefix from-fluent-bit-
auto_create_group On
```

But when running the above, the following error occurs:

```
{"data":{"filters.conf":"[FILTER]\n Name cri\n Match from-fluent-bit-kube.*\n Key_Name log\n Parser cri\n","parsers.conf":null},"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"data\":{\"filters.conf\":\"[FILTER]\\n Name cri\\n Match from-fluent-bit-kube.*\\n Key_Name log\\n Parser cri\\n\",\"output.conf\":\"[OUTPUT]\\n Name cloudwatch\\n Match *\\n region us-west-2\\n log_group_name fluent-bit-cloudwatch\\n log_stream_prefix from-fluent-bit-\\n auto_create_group On\\n\"},\"kind\":\"ConfigMap\",\"metadata\":{\"annotations\":{},\"name\":\"aws-logging\",\"namespace\":\"aws-observability\"}}\n"}}}
to:
Resource: "/v1, Resource=configmaps", GroupVersionKind: "/v1, Kind=ConfigMap"
Name: "aws-logging", Namespace: "aws-observability"
for: "fluentbit-config.yaml": admission webhook "0500-amazon-eks-fargate-configmaps-admission.amazonaws.com" denied the request: cri is not a supported filter. Please fix the logging configmap
```

**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

Contributor guide

Open the contributing guide

Research direction

Start with the EKS Fargate logging documentation and the supplied aws-logging ConfigMap examples, including filters.conf, parsers.conf, and fluentbit-config.yaml. Confirm the admission configuration accepts the cri filter and parser for Fargate logs; done means the documented cri configuration is no longer rejected as an unsupported filter.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.