[EKS] [request]: Nodelocal DNS Cache
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
**Tell us about your request**
I would like an officially documented and supported method for installing the Kubernetes Node Local DNS Cache Addon.
**Which service(s) is this request for?**
EKS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
Kubernetes clusters with a high request rate often experience high rates of failed DNS lookups. For example this affects us when using the AWS SDKs particularly with Alpine / musl-libc containers.
The _Nodelocal DNS Cache_ aims to resolve this (together with kernel patches in 5.1 to fix a conntrack race condition).
[Nodelocal DNS Addon](https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns/nodelocaldns)
Kubeadm is aiming to support Nodelocal-dns-cache in 1.15. [k/k #70707](https://github.com/kubernetes/kubernetes/issues/70707)
**Are you currently working around this issue?**
Retrying requests at the application level which fail due to DNS errors.
**Additional context**
Kubernetes DNS Issues include:
- Linux Kernel bug in netfilter conntrack (fixed in kernel 5.1) [1][2]
- Exacerbated by musl-libc behaviour of issuing parallel queries, which is widely used in Alpine Docker containers. musl-libc does not respect the resolv.conf `single-request` option and it appears this will not be changed [2][3]
- The AWS EC2 limit of 1024 Packets Per Second Per ENI [5]
**Attachments**
[0] https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Config.html#retryDelayOptions-property
[1] https://lkml.org/lkml/2019/2/28/707
[2] https://blog.quentin-machu.fr/2018/06/24/5-15s-dns-lookups-on-kubernetes/
[3] https://www.weave.works/blog/racy-conntrack-and-dns-lookup-timeouts
[4] https://www.openwall.com/lists/musl/2015/10/22/15
[5] https://docs.aws.amazon.com/vpc/latest/userguide/vpc-dns.html#vpc-dns-limits
Contributor guide
Research direction
Read the linked Kubernetes NodeLocal DNS Addon documentation and Kubernetes issue #70707 first, then compare them with the EKS request. Done means EKS has an officially documented and supported installation method for NodeLocal DNS Cache, including guidance relevant to the DNS lookup failures described here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, linux
- Domain
- cloud, infrastructure, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100