apple / apple/container

[Bug]: SERVFAIL resolving certain external hostnames from inside a container (default network), while host-side resolution of the same names succeeds

Open
#2,135 2 comments 0 reactions 0 assignees View on GitHub
reliability
Dominant language
Swift
Stars
49.9k
Forks
1.8k
Avg merge
1d 20h
Merged PRs (30d)
22

Description

### I have done the following

- [x] I have searched the existing issues
- [x] If possible, I've reproduced the issue using the 'main' branch of this project

### Steps to reproduce

```bash
# Any container on the default network, resolving via the vmnet gateway's DNS relay
container run --rm alpine sh -c '
nslookup google.com;
echo ---;
nslookup auth.docker.io;
echo ---;
nslookup registry-1.docker.io
'
```

Concretely reproduced via `docker buildx build` with the `docker-container` driver (buildx's `buildkit` container runs on the default network like any other container, so the same relay applies):

```bash
docker buildx create --name test --driver docker-container --use
printf '# syntax=docker/dockerfile:1.4\nFROM alpine\nRUN echo hi\n' > Dockerfile
docker buildx build .
# ERROR: failed to solve: failed to fetch anonymous token:
# Get "https://auth.docker.io/token?...": dial tcp: lookup auth.docker.io: no such host
```

### Problem description

**Actual behavior:** `nslookup` from inside a container against the default network's DNS relay (`192.168.64.1`, learned via DHCP) returns `SERVFAIL` for some external hostnames, while others resolve correctly through the exact same server in the same session:

```
$ nslookup google.com
Server: 192.168.64.1
Non-authoritative answer:
Name: google.com
Address: 172.217.19.238
Address: 2a00:1450:400f:808::200e

$ nslookup auth.docker.io
Server: 192.168.64.1
** server can't find auth.docker.io: SERVFAIL
** server can't find auth.docker.io: SERVFAIL

$ nslookup registry-1.docker.io
Server: 192.168.64.1
** server can't find registry-1.docker.io: SERVFAIL
Non-authoritative answer:
Name: registry-1.docker.io
Address: 2600:1f18:2148:bc00:e9ac:7b1d:e583:dcb1
Address: 2600:1f18:2148:bc01:a836:6254:ff65:cdf5
[... more AAAA records ...]
```

Note the pattern for `registry-1.docker.io`: the **AAAA (IPv6) query succeeds** with a full list of real addresses, while the **A (IPv4) query for the same name gets `SERVFAIL`**. For `auth.docker.io`, both query types return `SERVFAIL` with no answer at all.

**This is not a general network-reachability problem.** I isolated it carefully:

1. **Host-side resolution/pulling of the exact same names works perfectly** — `container image pull` and `docker pull` (via a Docker-compatible daemon layered on top of `container`) both succeed instantly for images from `registry-1.docker.io`, `docker.io`, `public.ecr.aws`, and `ghcr.io`. This code path resolves DNS on the macOS host directly, not through the guest's network stack.
2. **Only guest-side resolution through the vmnet gateway relay is affected.** Any process running *inside* a container that needs to resolve `auth.docker.io` itself (rather than have the host resolve it) hits this — which is exactly what buildx's `docker-container` driver does, since its `buildkit` container makes its own outbound requests from inside the guest VM.
3. It is consistent and reproducible across multiple fresh containers/builder instances, not a one-off flake — retried several times with the same result.

**Why I don't think this is the same issue as #656 / fixed by #1370:** #1370 fixed a *stale, hardcoded* DNS server baked into the builder container's config at creation time, which would go wrong after a system restart changed the network subnet. That's a "wrong/stale server" bug. This is different: the DNS server address here (`192.168.64.1`) is correct and current for the running network, and it directly answers plenty of other domains (`google.com`) correctly — it just returns `SERVFAIL` for a subset of real, resolvable hostnames, and inconsistently across A vs. AAAA for at least one of them.

**Where I looked in this repo:** I couldn't find any DNS relay/forwarding implementation for the default network's gateway DNS (`192.168.64.1`) anywhere in `Sources/Plugins/NetworkVmnet` or elsewhere in this repository. `Sources/DNSServer` / `Sources/APIServer/ContainerDNSHandler.swift` appear to implement a *different* thing — the host-side `container system dns` service for `*.test`-domain container-name lookups (documented in `docs/networking.md`), not the guest's default-network external-DNS relay. My best guess is that the default network's DNS relay is provided by `vmnet.framework` itself (a closed macOS system framework, via DHCP-advertised DNS), which would put the actual bug outside this repository's code — but filing here since this is the user-facing project people diagnosing "DNS doesn't work in my container" land on (as `#656`, `#402`, `#989`, `#856`, `#1693` already show), and the pattern (some names SERVFAIL, others succeed, host-side is fine) seemed worth documenting precisely in case it points at something fixable in how `container` configures or negotiates with vmnet, even if the ultimate root cause is a level below.

### Environment

- OS: macOS 26.6.1 (25G76)
- Xcode: 26.6 (17F113)
- Container: container CLI version 1.2.2 (build: release, commit: unspeci)

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the Alpine nslookup commands and the docker-container buildx driver. Then inspect Sources/Plugins/NetworkVmnet and compare it with Sources/DNSServer and Sources/APIServer/ContainerDNSHandler.swift; determine whether this repository configures the guest DNS relay or whether the behavior belongs to vmnet.framework. Done means identifying a repository-level cause or documenting that the issue is external.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, swift
Domain
cli, networking
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.