aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

bug(agent image mismatch): arm64 agent-image build breaks on x86 hosts after reboot — binfmt registration is ephemeral (structural fix: native ARM builder)

Offen
#571 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
agent-runtime bug enhancement infra-cdk P1
Vorherrschende Sprache
TypeScript
Sterne
143
Forks
46
Ø Merge
3 T. 10 Std.
Gemergte PRs (30 T.)
24

Beschreibung

### Component

Infra / build & deploy (agent image build; `mise //cdk:deploy`) + onboarding docs

### Describe the feature

Make the arm64 agent-image build survive a fresh/rebooted host — either by not depending on QEMU emulation at all (build natively on ARM), or by documenting a *persistent* binfmt registration. Today the only guidance (#431) is a one-shot manual `binfmt --install` that silently disappears on the next reboot or Docker-daemon restart.

### Use case

The agent image targets `linux/arm64` (Graviton). On an **x86_64** host, `mise //cdk:deploy` needs QEMU/binfmt emulation or the build dies with `exec /bin/sh: exec format error`.

#431 (closed) added a docs pre-empt step:

```
docker run --privileged --rm tonistiigi/binfmt --install arm64
```

But that command writes handlers into the kernel's `/proc/sys/fs/binfmt_misc/` — **runtime kernel state, not on-disk config**. It is wiped by:

- a reboot (a cloud dev box stopping/starting, or a fresh instance), and
- a Docker daemon restart.

So a deployer who followed #431 and got a green deploy comes back after a reboot to the *same* `exec format error`, with no doc hint that the registration is ephemeral. Live-hit on a cloud dev box: binfmt registered in an earlier session, gone after the box restarted, deploy failed again with the truncated `exec format error`.

This blocks any non-Graviton deployer intermittently and is the last remaining piece of the "make a fresh deploy succeed without gap-by-gap rework" arc (bootstrap least-privilege #403/#405/#408/#410 + docs #431).

### Proposed solution

Preferred (structural — removes the emulation dependency entirely):
- Build the agent image on a **native ARM64 builder** (Graviton runner / CodeBuild ARM fleet) so x86 hosts never touch binfmt. Wire it into `deploy.yml` / the deploy task so both CI and local x86 deploys get a native build.

Fallback (if emulation must stay for local x86):
- Document a **persistent** binfmt registration instead of the one-shot:
- Debian/Ubuntu: `sudo apt install -y qemu-user-static binfmt-support` (installs the `systemd-binfmt` service that re-applies `/etc/binfmt.d/` every boot).
- Amazon Linux: `sudo yum install -y qemu-user-static`, or a `@reboot`/systemd unit that re-runs the `tonistiigi/binfmt --install arm64` command.
- In `QUICK_START` + the `/setup` skill, add a one-liner: "this registration does **not** survive a reboot or Docker restart — use the persistent install, or a Graviton host, so you don't re-hit `exec format error`."

### Other information

- Related: #431 (docs pre-empt, closed — covered the manual step but not durability or the structural build) and the bootstrap least-privilege cluster #403/#405/#408/#410.
- Not a stack bug — the CDK/image is correct; this is a host-environment/build-topology gap.
- Surfaced while bringing the context-gated ECS compute substrate live (`--context compute_type=ecs`), but the `linux/arm64` image build (and therefore this blocker) is independent of the ECS vs AgentCore substrate choice.

### Acknowledgements

- [ ] I may be able to implement this feature

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne mit dem Einstiegspunkt `mise //cdk:deploy` und untersuche `deploy.yml`, `QUICK_START` und den `/setup`-Skill, um nachzuverfolgen, wie das arm64-Image auf x86-Hosts erstellt wird. Ermittle, ob die Build-Verdrahtung einen nativen ARM-Builder unterstützt oder dauerhafte binfmt-Hinweise benötigt. Als erledigt gilt die Aufgabe, wenn ein frisch eingerichteter oder neu gestarteter Host das arm64-Deployment ohne wiederkehrende `exec format error`-Fehler abschließen kann.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
aws, docker, typescript
Bereich
build-system, devops, documentation, infrastructure
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
38/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.