kiwix / kiwix/operations

Revisit Kube's DNS config?

Open
#776 3 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
15
Forks
5
PR merge metrics
No merged PRs in 30d

Description

Any pod's `resolv.conf` looks like this:

```
search {namespace}.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.32.0.10
options ndots:5
```

That's on purpose ([k8s DNS](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/)) so pods can communicate with one another using names in-namespace and out-of-namespace.
With pods being able to use subdomain, that can lead to names like cont1.poda.namespace.svc.cluster.local : 5 dots in this _local_ name. That's why the conf has `ndots: 5`.

The consequence of this is that in our containers, **every single unqualified domain lookup** (close to every connection) to an _external_ domain generates 4 DNS queries (just for IPv4). That's a lot of noise, a lot of wasted cpu, a lot of _internal_ traffic and a lot of wasted time in apps.

What can we do to minimize this?

- consider it not an issue (just wasted resources that are not in stress)
- We can use fully-qualified external domains: Use `s3.us-east-2.wasabisys.com.` instead of `s3.us-east-2.wasabisys.com`. Most our services uses constants for known external URLs.
- We can use fully-qualified (or just full-form) notation for our inter-pods comunication: Use `farm-ui-service.zimit.svc.cluster.local.` (or `farm-ui-service.zimit.svc.cluster.local`) instead of `farm-ui-service` and reduce `ndots`.
- We can use the in-pod environ to get IPs and ports of services and reduce `ndots`.
- Use only no-dots for _internal_ with `ndots: 0` and selectively increase (or use alternative) for cases where it's needed (inter-namespace comm).

I believe the last option is the easier from current situation.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the Kubernetes pod DNS configuration and the linked Kubernetes DNS documentation. Compare the proposed ndots and fully qualified naming options, then verify that the chosen approach reduces external DNS queries without breaking in-cluster service discovery.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
infrastructure, networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.