apple / apple/container

[Bug]: Bootstrap fails (configureDns/configureHosts) on images without writable /etc — blocks Redroid (Android-in-container)

Open
#1,737 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Swift
Stars
49.9k
Forks
1.8k
Avg merge
1d 20h
Merged PRs (30d)
22

Description

### Description

`container run` fails at the bootstrap phase for OCI images that don't have a writable / standard-Linux `/etc` layout — specifically, Android-based container images like [Redroid](https://github.com/remote-android/redroid-doc). The failure happens before container init is reached:

```
Error: failed to bootstrap container (cause: "internalError: "failed to bootstrap container redroid-poc (cause: "unknown: "internalError: "configureDns""")"")
```

Bypassing DNS with `--no-dns` advances the failure to the next pre-init step:

```
Error: failed to bootstrap container (cause: "internalError: "failed to bootstrap container redroid-poc (cause: "unknown: "internalError: "configureHosts""")"")
```

No further bypass flag is exposed in `container run --help` to skip `configureHosts`, so these images can't be launched at all today.

### Use case context

I'm evaluating Apple Container as the per-container Android runtime for an agent orchestrator (replaces our current Docker Desktop setup, which can't expose KVM to containers). The infrastructure stack is:

- **macOS 26.5.1 (Tahoe)** + **M4 Pro** (so VZ nested virtualization is available)
- `apple/container` `1.0.0` (signed installer)
- Custom Linux kernel **built from `apple/containerization` `kernel/config-arm64`** (default Kata kernel has `CONFIG_KVM=n`)
- Kernel config additionally extended with `CONFIG_ANDROID_BINDER_IPC=y`, `CONFIG_ANDROID_BINDERFS=y`, `CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder"` so Redroid can mount binderfs

With that kernel and `container run --virtualization --kernel --cap-add ALL ` — **everything works**. KVM and binder are both fully usable inside the micro-VM:

```
$ container run --rm --virtualization --kernel ./vmlinux --cap-add ALL alpine sh -c '
ls -l /dev/kvm
dmesg | grep -i kvm
mkdir /tmp/b && mount -t binder binder /tmp/b && ls /tmp/b
'
crw------- 1 root root 10, 232 Jan 1 1970 /dev/kvm
[ 0.020224] kvm [1]: nv: 568 coarse grained trap handlers
[ 0.020258] kvm [1]: IPA Size Limit: 40 bits
[ 0.020656] kvm [1]: Hyp nVHE mode initialized successfully
binder binder-control features hwbinder vndbinder
```

So the foundation is solid — the *only* thing blocking the Redroid path right now is `container`'s pre-init bootstrap.

### Reproduction

```bash
# Reproduces verbatim on macOS 26.5.1 / M4 Pro with container 1.0.0:

# 1. Custom kernel (from apple/containerization with the binder/ashmem additions noted above).
# 2. Pull Redroid (~1.5 GB uncompressed):
container image pull docker.io/redroid/redroid:14.0.0_64only-latest

# 3. Run — fails at configureDns:
container run --detach --name redroid-poc \
--virtualization --kernel ~/Documents/GitHub/containerization/kernel/vmlinux \
--cap-add ALL --memory 4g --cpus 4 \
docker.io/redroid/redroid:14.0.0_64only-latest
# Error: failed to bootstrap container (cause: ... "configureDns" ...)

# 4. With --no-dns, fails at the next step:
container run --detach --name redroid-poc \
--virtualization --kernel ~/Documents/GitHub/containerization/kernel/vmlinux \
--cap-add ALL --memory 4g --cpus 4 --no-dns \
docker.io/redroid/redroid:14.0.0_64only-latest
# Error: failed to bootstrap container (cause: ... "configureHosts" ...)
```

A standard `debian:bookworm` image with the same flag set runs fine, so the failure is specific to images that don't follow Apple Container's assumed pre-init filesystem layout.

### Expected behavior / proposed fix

Either of the following would unblock the Redroid path (and likely other Android-runtime-style images):

1. **A `--no-host-config` / `--minimal-bootstrap` flag** that skips _all_ pre-init mutations the runtime applies to the image rootfs (DNS, hosts, /etc/resolv.conf, anything else under that umbrella). `--no-dns` already exists; a sibling for hosts + everything else needed for these images would be the lightest-touch option.
2. **A best-effort fallback** in the bootstrap path that skips a configuration step when the target file isn't writable / the rootfs isn't standard, with a single warning log line instead of a hard fail.
3. **Documentation** of the supported workaround — e.g. a custom `--init-image` that absorbs the configuration steps so the rootfs is left alone. Today `--init-image` exists but I couldn't find docs on what shape is required to satisfy `configureDns`/`configureHosts`.

### What's not the issue

- Kernel — works, KVM + binder verified above.
- Capabilities — `--cap-add ALL` is set.
- Image arch — confirmed pulled as `linux/arm64`.
- Image integrity — Redroid runs on plain Docker on Linux hosts with the same image tag, so it isn't a malformed OCI bundle.

### Environment

| | |
|---|---|
| Host | MacBook Pro M4 Pro |
| macOS | 26.5.1 (build 25F80) |
| Apple Container | 1.0.0 (commit ee848e3) |
| Kernel | Built from `apple/containerization` `kernel/config-arm64` (Linux 6.18.5) with `CONFIG_KVM=y`, `CONFIG_ANDROID_BINDER_IPC=y`, `CONFIG_ANDROID_BINDERFS=y` |
| Image | `docker.io/redroid/redroid:14.0.0_64only-latest` |

Contributor guide

Open the contributing guide

Research direction

Start at the bootstrap path where configureDns and configureHosts run, and compare the behavior with and without --no-dns on the Redroid image. Determine how non-writable or non-standard /etc layouts should be handled, then verify that the image can reach container init without the bootstrap failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, linux, macos, swift
Domain
cli, infrastructure, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.