aws / aws/aws-nitro-enclaves-cli

`nitro-cli build-enclave` fails with E48 (linuxkit image not found) on Docker v27+

Open
#746 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
156
Forks
99
PR merge metrics
No merged PRs in 30d

Description

## Description

`nitro-cli build-enclave --docker-uri ` fails when the Docker daemon is v27 (API 1.54). The Docker image is successfully built and exists in the daemon, but linuxkit cannot resolve it from its local cache and attempts to pull from Docker Hub instead.

This works correctly on Docker v24 (API 1.41) — same `nitro-cli` version, same eif-builder container, same workflow.

## Environment

- **nitro-cli version:** 1.3.1 (Amazon Linux 2023, `aws-nitro-enclaves-cli-1.3.1-0.amzn2023.x86_64`)
- **Docker (working):** v24.0.x, API 1.41 (GitHub-hosted ubuntu-latest runner)
- **Docker (failing):** v27.x, API 1.54 (self-hosted runner, overlay2 storage driver, containerd-snapshotter disabled)
- **Host OS:** Ubuntu 22.04
- **Runner setup:** nitro-cli runs inside an Amazon Linux container with the host Docker socket mounted (`-v /var/run/docker.sock:/var/run/docker.sock`)

## Steps to reproduce

1. On a host with Docker v27+, build a Docker image:
```bash
docker build -t enclave:latest -f path/to/Dockerfile .
```

2. Verify the image exists:
```bash
docker images enclave
# Shows enclave:latest
```

3. Run nitro-cli inside an Amazon Linux container with the socket mounted:
```bash
docker run \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /output:/output \
amazonlinux-with-nitro-cli \
nitro-cli build-enclave --docker-uri enclave:latest --output-file /output/enclave.eif
```

4. nitro-cli outputs:
```
Start building the Enclave Image...
Using the locally available Docker image...
Linuxkit reported an error while creating the customer ramfs:
"Add init containers:
Process init image: docker.io/library/enclave:latest
Image docker.io/library/enclave:latest not found in local cache, pulling
Failed to build init tarball from docker.io/library/enclave:latest:
Could not pull image docker.io/library/enclave:latest:
error getting manifest for trusted image docker.io/library/enclave:latest:
GET https://index.docker.io/v2/library/enclave/manifests/latest: UNAUTHORIZED"
```

## Analysis

- nitro-cli's bollard client connects to Docker v27 successfully (the `docker build` via `--docker-dir` works, and "Using the locally available Docker image" confirms the image is found via the API).
- The failure is in **linuxkit**, which resolves images through Docker's internal storage rather than the socket API. Docker v27 changed the internal storage layout, and linuxkit cannot find images there.
- Disabling the containerd snapshotter (`"containerd-snapshotter": false` in `daemon.json`) does not fix the issue — linuxkit fails regardless of storage backend on Docker v27.
- The same workflow succeeds on Docker v24 with identical nitro-cli version and container setup.

## Expected behavior

`nitro-cli build-enclave` should work with Docker v27+ since it is the current stable Docker release. Linuxkit should resolve locally available images through the Docker socket API rather than accessing internal storage directly.

## Workaround

Pin Docker to v24.0.x on the host:
```bash
sudo apt-get install -y --allow-downgrades \
docker-ce=5:24.0.9-1~ubuntu.22.04~jammy \
docker-ce-cli=5:24.0.9-1~ubuntu.22.04~jammy
sudo apt-mark hold docker-ce docker-ce-cli
```

Contributor guide

Open the contributing guide

Research direction

Start at the nitro-cli build-enclave entry point and trace the bollard Docker image lookup into the linuxkit invocation. Reproduce the provided workflow with Docker v24 and v27, then verify the fix by producing /output/enclave.eif on Docker v27 without a registry pull.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, rust
Domain
cli, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.