nextflow-io / nextflow-io/nextflow

ARM images for AWS public registry

Open
#4,314 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

software/docker stale
Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

Bug report

There's a chance I'm doing something boneheaded here, but I am trying to pull the latest seqera-labs AWS batch docker images from public.ecr.aws/seqera-labs/nextflow:latest from an AWS c6g linux/arm64/8 image, and getting linux/amd64 images.

Expected behavior and actual behavior

The seqera-labs gallery suggests that arm images are available. From an arm64 machine, I am invoking this build with the Dockerfile below

docker build --platform linux/arm64 .

I would expect that to select the arm images (I couldn't see anything from the gallery that indicated architecture) but I'm assuming the docker build grabs the right one.

ARG VERSION=latest
FROM public.ecr.aws/seqera-labs/nextflow:${VERSION} AS build


RUN yum update -y \
 && yum install -y \
    unzip \
 && yum clean -y all
RUN rm -rf /var/cache/yum

# install awscli v2
RUN curl -s "https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip" -o "/tmp/awscliv2.zip" \
 && unzip -q /tmp/awscliv2.zip -d /tmp \
 && /tmp/aws/install -b /usr/bin \
 && rm -rf /tmp/aws*

# install a custom entrypoint script that handles being run within an AWS Batch Job
COPY nextflow.aws.sh /opt/bin/nextflow.aws.sh
RUN chmod +x /opt/bin/nextflow.aws.sh

WORKDIR /opt/work
ENTRYPOINT ["/opt/bin/nextflow.aws.sh"]

When I don't explicitly add the target, I get an architecture error - it clearly pulled the wrong version.

$ docker build .
Sending build context to Docker daemon  7.168kB
Step 1/9 : ARG VERSION=latest
Step 2/9 : FROM public.ecr.aws/seqera-labs/nextflow:${VERSION} AS build
latest: Pulling from seqera-labs/nextflow
8be3d01330d7: Pull complete
05200b11ae6e: Pull complete
4d4e2f332acf: Pull complete
f29f4675beae: Pull complete
1f0835b691b9: Pull complete
85af6e7a5c27: Pull complete
4f8bff6a50cd: Pull complete
Digest: sha256:01bdca61259a1a431c9d8534c0bb8353d413cce8f9638bb8d4e7b854cb5ce43c
Status: Downloaded newer image for public.ecr.aws/seqera-labs/nextflow:latest
 ---> a1c176ab3423
Step 3/9 : RUN yum update -y  && yum install -y     unzip  && yum clean -y all
 ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
 ---> Running in d0d491444ad2
exec /bin/sh: exec format error
The command '/bin/sh -c yum update -y  && yum install -y     unzip  && yum clean -y all' returned a non-zero code: 1

When I force the architecture explicitly, docker warns me

WARNING: Pulled image with specified platform (linux/arm64), but the resulting image's configured platform (linux/amd64) does not match.
This is most likely caused by a bug in the build system that created the fetched image (public.ecr.aws/seqera-labs/nextflow:23.04.3).
Please notify the image author to correct the configuration.
Steps to reproduce the problem

On an arm64 machine, using the Dockerfile above, try

docker build --platform linux/arm64 .

Program output

See above

Environment
  • Linux aws c6g instance
  • 6.1.34-59.116.amzn2023.aarch64 # 1 SMP Thu Jun 29 18:11:49 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
  • Docker version 20.10.23, build 7155243
Additional context

Hoping it's something dumb I've done. I'd be happy to try to build the image from scratch - looking at the source right now to figure out how it was probably built.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Dockerfile and the docker build --platform linux/arm64 . command shown in the report, then inspect how the public.ecr.aws/seqera-labs/nextflow image is published. Verify the image metadata and ARM64 build behavior, including the configured platform warning. Done means an ARM64 build selects a usable ARM64 base image without an architecture mismatch.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker
Domain
cloud, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.