containers / containers/podman-compose
aardvark-dns exits during `podman-compose up` for 4-6 service projects on Windows/WSL2 — does NOT reproduce with raw `podman run` (incl. parallel)
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 622
- PR merge metrics
- No merged PRs in 30d
Description
Summary
podman-compose up consistently fails for a specific class of multi-container projects with:
Error: unable to start container "<id>": netavark (exit code 1): error while applying dns entries: aardvark-dns exited unexpectedly without error messageAffected containers stay stuck in Created. The crash leaves no aardvark-dns log line — netavark's wrapper message is the only signal.
Crucially: I cannot reproduce this with the podman CLI alone. A standalone reproducer that issues the equivalent podman network create + podman run calls — both sequentially and concurrently — succeeds every time on the same machine. The failure only appears when podman-compose drives the bring-up, which is why I'm filing here rather than at containers/podman. (Originally reported at containers/podman#<original-issue-number>; a maintainer confirmed podman-compose is out of scope there and asked for a raw-CLI repro, which is included below.)
Environment
- Host OS: Windows 10 Pro 25H2 build 26200.8457
- podman: 5.8.2 (WSL2 backend)
- podman-compose: version 1.5.0
[repro-logs.zip](https://github.com/user-attachments/files/28081488/repro-logs.zip)
[repro-logs-parallel.zip](https://github.com/user-attachments/files/28081487/repro-logs-parallel.zip) - Network backend: netavark
- DNS: aardvark-dns (bundled with podman 5.x)
- Rootless: yes
Minimal reproduction
repro-compose.yaml:
name: repro-aardvark-bug
services:
postgres:
image: docker.io/postgres:16.13-alpine
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD: app
POSTGRES_DB: app
clickhouse:
image: docker.io/clickhouse/clickhouse-server:26.3
environment:
CLICKHOUSE_USER: app
CLICKHOUSE_PASSWORD: app
redis:
image: docker.io/library/redis:7-alpine
minio:
image: docker.io/minio/minio:latest
command: ["server", "/data"]
environment:
MINIO_ROOT_USER: minioadmin
MINIO_ROOT_PASSWORD: minioadminpodman-compose -f repro-compose.yaml up -d(From a clean state: podman stop -a; podman rm -af; podman volume prune -f; podman network prune -f.)
Observed
postgres+clickhousereachUp; the remaining services fail to start with theaardvark-dns exited unexpectedlyerror and stayCreated.- Retrying
podman-compose up, tearing down and re-running, host reboot, and fullpodman machine rm -f+init+startall produce the same failure. - The failure clusters on 4-6 service projects that include bare
postgres+clickhouseservice names. Single-container and 3-service projects work; 10+ service projects with prefixed service names (e.g.lf-postgres) also work.
Expected
All services reach Up; aardvark-dns registers DNS entries for every service on the project's default network.
Key diagnostic — raw podman CLI does NOT reproduce it
I built a reproducer using only podman CLI (no podman-compose), --log-level=debug on every call, fresh custom network per test, full teardown between tests. 8 of 8 tests pass, including the exact shapes that fail under podman-compose:
Test | Shape | Result
-- | -- | --
1× alpine, default net | sanity | PASS
1× alpine, custom net | sanity | PASS
postgres + clickhouse (bare names) | 2-svc | PASS
postgres + clickhouse + redis + minio (bare) | 4-svc | PASS
+ minio-init (matches a real failing compose) | 5-svc | PASS
same 4-svc, prefixed names (lf-*) | control | PASS
4-svc bare, started in parallel | concurrency | PASS
4-svc bare, parallel, ×5 back-to-back | flake-hunt | PASS
The parallel tests fire 4 concurrent podman run -d calls into the same fresh network (4-5 aardvark registrations inside a ~1s window) and still succeed. So the trigger is not raw podman run, bare service names, service count, or concurrent DNS registration in isolation — it's something podman-compose layers on top of podman run.
What podman-compose does that the raw-CLI repro did not
Candidate triggers, for whoever investigates:
- Healthchecks — compose
healthcheck:adds--health-cmd/--health-interval; healthcheck threads do in-container DNS lookups. - Bind-mount volumes — host-path mounts (on WSL2 these route through 9p; first-time setup can race with container start).
- Port publishing —
-pfor every exposed port. - Restart policies —
--restart=unless-stopped(restart loops re-register DNS for the same name). --env-filerather than per--e.- Container labels —
io.podman.compose.*. - Network creation options — any
--opt/--subnet/--ipam-driverpodman-compose passes when creating the project's default network. - Whether podman-compose uses
podman play kubevs directpodman runfor any path.
I'm happy to extend the CLI reproducer with any one of these (healthchecks + bind-mounts + ports + restart are present on every failing project) to bisect which one trips aardvark — just let me know which you'd like first, or whether a --log-level=debug trace of the actual failing podman-compose up (showing the exact podman invocations issued) would be more useful.
What I tried (none fixed it)
- Full Windows reboot.
podman machine stop/start.podman machine rm -f+init+start(verified fresh viapodman info).podman system prune -af --volumes;podman network prune -fbetween attempts.- Manual
podman network create+ container-on-network — works standalone. - DNS warmup (create+destroy a network with one container before the failing compose) and a single retry-after-warmup — both still fail under compose.
Attachments
repro-logs.zip/repro-logs-parallel.zip— full--log-level=debugoutput per step from the raw-CLI reproducer (all PASS), pluspodman infoandpodman --version.- A
--log-level=debugcapture of the failingpodman-compose upcan be provided on request.
Diagnostic-improvement note
aardvark-dns crashes "without error message" — the only signal is netavark's exit-code-1 wrapper. A more verbose aardvark-dns log mode, or capturing the aardvark-dns subprocess stderr into the podman/compose log, would make this class of issue much easier to diagnose.
Contributor guide
Assessment
This issue has not been assessed yet.