agent-substrate / agent-substrate/substrate

Implement CoreDNS ConfigMap integration in the dns-controller

Open
#348 1 comment 0 reactions 1 assignee Claimed by @bowei View on GitHub
area/network kind/feature
Dominant language
Go
Stars
1.8k
Forks
316
Avg merge
2d 43m
Merged PRs (30d)
287

Description

## Summary

The `dns-controller` README at `cmd/atenet/internal/dns/README.md` lists two integration paths for wiring substrate's stub resolver into the cluster's DNS:

```
## Integration

* CoreDNS: Update CoreDNS ConfigMap to add the stub resolver.
* GKE DNS: Update the GKE DNS ConfigMap to add the stub resolver.
```

The GKE path lives in `reconcileKubeDNSConfig` today (writes `stubDomains` JSON into `kube-system/kube-dns`). Implementing the CoreDNS path would let substrate's actor hostnames resolve on any cluster using a stock CoreDNS Corefile — kubeadm-based clusters (kind, k3s, vanilla self-managed), EKS, AKS, on-prem, and GKE installs that don't surface the kube-dns compatibility ConfigMap.

## Motivation

On a stock kubeadm cluster the cluster DNS is CoreDNS configured by a `kube-system/coredns` ConfigMap whose `Corefile` key holds the CoreDNS DSL:

```yaml
data:
Corefile: |
.:53 {
errors
kubernetes cluster.local in-addr.arpa ip6.arpa { ... }
forward . /etc/resolv.conf { ... }
cache 30
loop
reload
loadbalance
}
```

After installing substrate, the `dns-controller` logs:

```
kube-dns ConfigMap not found in kube-system namespace, skipping stub resolver configuration
```

…and the cluster's CoreDNS never learns where to forward `actors.resources.substrate.ate.dev`. Adding the CoreDNS path closes the loop for these clusters without operator intervention.

## Proposed implementation

A peer to `reconcileKubeDNSConfig` that targets `kube-system/coredns`, gated by a new flag on `atenet dns`:

1. **Selection.** Add `--cluster-dns-style=kube-dns|coredns` to the `atenet dns` subcommand, defaulting to `kube-dns` so existing GKE deployments are unaffected. Operators on kubeadm-based clusters pass `--cluster-dns-style=coredns`. Auto-detection can come later as a separate enhancement if useful — keeping the first cut explicit is simpler to review and reason about.

2. **Editing the Corefile idempotently.** Use a CoreDNS Corefile parser (e.g. `github.com/coredns/caddy/caddyfile`) to load the AST, splice in / update / remove a substrate-owned server block, and serialize back. Avoids text/regex munging on a shared cluster-infrastructure ConfigMap.

3. **Stub zone to insert.** A peer server block sibling to the existing `.:53`, marked so future reconciles can find and update it idempotently. Conceptually:

```
# managed by substrate dns-controller
actors.resources.substrate.ate.dev:53 {
errors
cache 30
forward .
}

.:53 { ... } # untouched
```

4. **Reload.** Write the updated ConfigMap. Stock kubeadm CoreDNS uses the `reload` plugin which watches the mounted Corefile and reloads on change, so no extra signaling needed.

5. **Removal.** On uninstall (or when the substrate CoreDNS Service goes away), remove the substrate-owned block cleanly so the cluster Corefile returns to its prior state.

## Risk surface

`kube-system/coredns` is shared cluster infrastructure — the implementation should:

- Touch only the substrate-owned server block; never modify the default `.:53` block.
- Be idempotent — repeated reconciles must converge to the same Corefile.
- Tolerate a hand-edited Corefile elsewhere in the file — leave non-substrate content alone.
- Fail soft on a malformed Corefile (log + skip a reconcile rather than rewrite).

## Repro on kind

```bash
./hack/install-ate-kind.sh
# wait for substrate components to come up

# Create an actor (any WorkerPool / ActorTemplate flow) and attempt to dial
# .actors.resources.substrate.ate.dev from a pod -> NXDOMAIN.

kubectl get cm -n kube-system coredns -o yaml
# Corefile has no substrate stub.

kubectl -n ate-system logs deploy/dns -c dns-controller
# "kube-dns ConfigMap not found in kube-system namespace, skipping stub resolver configuration"
```

## Contribution

Happy to take this on if the direction sounds right.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.