Building FCOS with qemu-user-static for different architectures

Feature Request

From my point of view, support for different architectures besides x86_64 is becoming increasingly important but it seems that the coreos-assembler container image does not support that yet. This means that one has to build the container image from scratch for the target architecture for any other architecture than x86_64, even though one might lack a proper build machine running on aarch64 for example.

At this stage not many options are left, one of them is using qemu-user-static but it is not supported out of the box and requires changes to coreos-assembler code base. An example implementation of what I am trying to achieve for aarch64 can be seen here, even though it does not work due to some limitations in coreos-assembler.

Desired Feature

I would love to see two things:

1. Add support for qemu-user-static that does not require patching of the coreos-assembler source code

2. Provide a multi-arch container image for coreos-assembler on quay.io that can be used by other to build their own custom FCOS on different architectures

Example Usage

This boils down to the following:

1. On any x86_64 machine build coreos-assembler from scratch:

sudo podman run --rm --pull always --privileged registry.hub.docker.com/multiarch/qemu-user-static:latest --reset -p yes
podman build --arch ${COREOS_ARCHITECTURE} -f target/src/coreos-assembler/Dockerfile -t localhost/coreos-assembler target/src/coreos-assembler/

Where:

• ${COREOS_ARCHITECTURE} - Target architecture such as aarch64 or s390x

2. Use the existing coreos-assembler container image if no rebuild is required:

sudo podman run --rm --pull always --privileged registry.hub.docker.com/multiarch/qemu-user-static:latest --reset -p yes
podman run \
    --rm -ti --security-opt label=disable --privileged \
    --arch ${COREOS_ARCHITECTURE} \
    --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 \
    -v ${PWD}/target/build/cosa:/srv/ --device /dev/kvm --device /dev/fuse \
    --tmpfs /tmp -v /var/tmp:/var/tmp --name cosa \
    quay.io/coreos-assembler/coreos-assembler:latest "${CMD}"

Where:

• ${COREOS_ARCHITECTURE} - Target architecture such as aarch64 or s390x
• ${CMD} - Some cosa command

Other Information

None.

Start with target/src/coreos-assembler/Dockerfile and the linked example Makefile, then trace how the container image is built and invoked for non-x86_64 architectures. Done means qemu-user-static support works without patching coreos-assembler and a usable multi-architecture image is published on quay.io.