[EKS][request]: enable JSON Format Control Plane Logs
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
I would like to request that AWS enable JSON format logging for the EKS Control Plane components. JSON format logging was released in version 1.19 (https://kubernetes.io/blog/2020/09/04/kubernetes-1-19-introducing-structured-logs/)
**Which service(s) is this request for?**
EKS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
The problem I'm trying to solve is making available easily parseable logs from EKS within CloudWatch Logs and CloudWatch Logs Insights.
Parsing logs via regex is _not_ ideal, both for individual customers or across all users of EKS. We are expirementing with the CloudWatch Insights parser below:
```
fields @timestamp, @message |
parse @message /(?\w\d{4})\s(?[0-1]?[0-9]|2[0-3]):(?[0-5][0-9]):(?[0-5][0-9]).(?\d{6})\s+\d+\s+(?\w+\.\w+):(?\d+)]\s+(?.*)/ |
display level, hour, minute, second, microsecond, file, linenumber, message
```
**Are you currently working around this issue?**
Currently we parse logs using glob syntax.
**Additional context**
I've confirmed that EKS Control Plane components (in particular, the scheduler) can produce JSON format logs. My method of confirming this is by running the following command: `minikube start --extra-config scheduler.logging-format=json`. An example of JSON format logs is below:
```
colinjohnson@cjohnson07 vpc % kubectl logs kube-scheduler-minikube --namespace kube-system
I0730 02:28:45.021355 1 serving.go:347] Generated self-signed cert in-memory
{"ts":1627612129283.1523,"msg":"Starting RequestHeaderAuthRequestController\n","v":0}
{"ts":1627612129283.3477,"msg":"Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file\n","v":0}
{"ts":1627612129283.4307,"msg":"Waiting for caches to sync for RequestHeaderAuthRequestController\n","v":0}
```
**Attachments**
Contributor guide
Research direction
Start by reviewing the Kubernetes 1.19 structured-logging announcement and the EKS control-plane logging documentation, then compare the minikube scheduler example in the issue with EKS behavior. Done means EKS exposes selectable JSON-formatted control-plane logs that remain usable in CloudWatch Logs and Logs Insights.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes
- Domain
- cloud, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 20/100