aws / aws/containers-roadmap

[EKS][request]: enable JSON Format Control Plane Logs

Open
#1,460 0 comments 14 reactions 0 assignees View on GitHub
EKS Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**

I would like to request that AWS enable JSON format logging for the EKS Control Plane components. JSON format logging was released in version 1.19 (https://kubernetes.io/blog/2020/09/04/kubernetes-1-19-introducing-structured-logs/)

**Which service(s) is this request for?**

EKS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**

The problem I'm trying to solve is making available easily parseable logs from EKS within CloudWatch Logs and CloudWatch Logs Insights.

Parsing logs via regex is _not_ ideal, both for individual customers or across all users of EKS. We are expirementing with the CloudWatch Insights parser below:

```
fields @timestamp, @message |
parse @message /(?\w\d{4})\s(?[0-1]?[0-9]|2[0-3]):(?[0-5][0-9]):(?[0-5][0-9]).(?\d{6})\s+\d+\s+(?\w+\.\w+):(?\d+)]\s+(?.*)/ |
display level, hour, minute, second, microsecond, file, linenumber, message
```

**Are you currently working around this issue?**

Currently we parse logs using glob syntax.

**Additional context**

I've confirmed that EKS Control Plane components (in particular, the scheduler) can produce JSON format logs. My method of confirming this is by running the following command: `minikube start --extra-config scheduler.logging-format=json`. An example of JSON format logs is below:

```
colinjohnson@cjohnson07 vpc % kubectl logs kube-scheduler-minikube --namespace kube-system
I0730 02:28:45.021355 1 serving.go:347] Generated self-signed cert in-memory
{"ts":1627612129283.1523,"msg":"Starting RequestHeaderAuthRequestController\n","v":0}
{"ts":1627612129283.3477,"msg":"Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file\n","v":0}
{"ts":1627612129283.4307,"msg":"Waiting for caches to sync for RequestHeaderAuthRequestController\n","v":0}
```

**Attachments**

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the Kubernetes 1.19 structured-logging announcement and the EKS control-plane logging documentation, then compare the minikube scheduler example in the issue with EKS behavior. Done means EKS exposes selectable JSON-formatted control-plane logs that remain usable in CloudWatch Logs and Logs Insights.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes
Domain
cloud, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.