docker / docker/buildx

segmentation fault cross-building target=amd64 host=arm64

Open
#2,028 44 comments 14 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/qemu
Dominant language
Go
Stars
4.5k
Forks
682
Avg merge
2d 14h
Merged PRs (30d)
29

Description

Contributing guidelines
I've found a bug and checked that ...
  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem
Description

I have an Apple silicon (M2) MacOS 13.5.1 laptop. On it I installed UTM 4.3.5. I made a VM running Ubuntu 22.04.3 as the guest. In that guest I installed Docker Engine 24.0.5. In that guest docker buildx build --platform linux/arm64,linux/ppc64le succeeds but asking for linux/amd64 makes the go comipler crash with segmentation fault.

Expected behaviour

Cross-platform builds work.

Actual behaviour

The go compiler crashes multiple times with a segmentation fault.

Buildx version

github.com/docker/buildx v0.11.2 9872040

Docker info
Client: Docker Engine - Community
 Version:    24.0.5
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.20.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 3
  Running: 2
  Paused: 0
  Stopped: 1
 Images: 6
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-82-generic
 Operating System: Ubuntu 22.04.3 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 15.59GiB
 Name: ubu22b
 ID: a0e3aa8f-77da-4014-8f91-f112c4b84b14
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: mspreitz
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
Builders list
NAME/NODE      DRIVER/ENDPOINT             STATUS  BUILDKIT             PLATFORMS
kubestellar    docker-container                                         
  kubestellar0 unix:///var/run/docker.sock running v0.12.1              linux/arm64
nice_allen *   docker-container                                         
  nice_allen0  unix:///var/run/docker.sock running v0.12.1              linux/arm64
default        docker                                                   
  default      default                     running v0.11.6+0a15675913b7 linux/arm64
Configuration
FROM golang:1.20 as builder

RUN mkdir -p /go/src/github.com/openshift && \
    cd /go/src/github.com/openshift && \
    git clone https://github.com/openshift/eventrouter

RUN cd /go/src/github.com/openshift/eventrouter && go mod tidy && CGO_ENABLED=0 go build .

FROM alpine:3.14 as run
WORKDIR /app
RUN apk update --no-cache && apk add ca-certificates
COPY --from=builder /go/src/github.com/openshift/eventrouter/eventrouter /app/eventrouter
USER nobody:nobody
CMD ["/bin/sh", "-c", "/app/eventrouter -v 3 -logtostderr"]

I built with the following command.

docker buildx build -t quay.io/mspreitz/eventrouter:latest --platform linux/amd64,linux/arm64,linux/ppc64le --push .
Build logs
That buildx command produced the following output.

[+] Building 231.5s (27/29)                                                 docker-container:nice_allen
 => [internal] load build definition from Dockerfile                                               0.0s
 => => transferring dockerfile: 738B                                                               0.0s
 => [linux/ppc64le internal] load metadata for docker.io/library/alpine:3.14                       0.2s
 => [linux/amd64 internal] load metadata for docker.io/library/alpine:3.14                         0.2s
 => [linux/amd64 internal] load metadata for docker.io/library/golang:1.20                         0.4s
 => [linux/ppc64le internal] load metadata for docker.io/library/golang:1.20                       0.2s
 => [linux/arm64 internal] load metadata for docker.io/library/alpine:3.14                         0.2s
 => [linux/arm64 internal] load metadata for docker.io/library/golang:1.20                         0.4s
 => [internal] load .dockerignore                                                                  0.0s
 => => transferring context: 2B                                                                    0.0s
 => CACHED [linux/ppc64le builder 1/3] FROM docker.io/library/golang:1.20@sha256:741d6f9bcab77844  0.0s
 => => resolve docker.io/library/golang:1.20@sha256:741d6f9bcab778441efe05c8e4369d4f8ff56c9a635a9  0.0s
 => [linux/ppc64le run 1/4] FROM docker.io/library/alpine:3.14@sha256:0f2d5c38dd7a4f4f733e688e3a6  0.0s
 => => resolve docker.io/library/alpine:3.14@sha256:0f2d5c38dd7a4f4f733e688e3a6733cb5ab1ac6e3cb46  0.0s
 => CACHED [linux/amd64 builder 1/3] FROM docker.io/library/golang:1.20@sha256:741d6f9bcab778441e  0.0s
 => => resolve docker.io/library/golang:1.20@sha256:741d6f9bcab778441efe05c8e4369d4f8ff56c9a635a9  0.0s
 => [linux/arm64 run 1/4] FROM docker.io/library/alpine:3.14@sha256:0f2d5c38dd7a4f4f733e688e3a673  0.0s
 => => resolve docker.io/library/alpine:3.14@sha256:0f2d5c38dd7a4f4f733e688e3a6733cb5ab1ac6e3cb46  0.0s
 => CACHED [linux/arm64 builder 1/3] FROM docker.io/library/golang:1.20@sha256:741d6f9bcab778441e  0.0s
 => => resolve docker.io/library/golang:1.20@sha256:741d6f9bcab778441efe05c8e4369d4f8ff56c9a635a9  0.0s
 => [linux/amd64 run 1/4] FROM docker.io/library/alpine:3.14@sha256:0f2d5c38dd7a4f4f733e688e3a673  0.0s
 => => resolve docker.io/library/alpine:3.14@sha256:0f2d5c38dd7a4f4f733e688e3a6733cb5ab1ac6e3cb46  0.0s
 => CACHED [linux/arm64 run 2/4] WORKDIR /app                                                      0.0s
 => CACHED [linux/arm64 run 3/4] RUN apk update --no-cache && apk add ca-certificates              0.0s
 => [linux/ppc64le builder 2/3] RUN mkdir -p /go/src/github.com/openshift &&     cd /go/src/gith  11.4s
 => CACHED [linux/ppc64le run 2/4] WORKDIR /app                                                    0.0s
 => CACHED [linux/ppc64le run 3/4] RUN apk update --no-cache && apk add ca-certificates            0.0s
 => [linux/arm64 builder 2/3] RUN mkdir -p /go/src/github.com/openshift &&     cd /go/src/github  11.7s
 => CACHED [linux/amd64 run 2/4] WORKDIR /app                                                      0.0s
 => CACHED [linux/amd64 run 3/4] RUN apk update --no-cache && apk add ca-certificates              0.0s
 => [linux/amd64 builder 2/3] RUN mkdir -p /go/src/github.com/openshift &&     cd /go/src/github  10.7s
 => ERROR [linux/amd64 builder 3/3] RUN cd /go/src/github.com/openshift/eventrouter && go mod t  220.2s
 => CANCELED [linux/ppc64le builder 3/3] RUN cd /go/src/github.com/openshift/eventrouter && go   219.6s
 => [linux/arm64 builder 3/3] RUN cd /go/src/github.com/openshift/eventrouter && go mod tidy &&   64.1s
 => [linux/arm64 run 4/4] COPY --from=builder /go/src/github.com/openshift/eventrouter/eventroute  0.1s
------
 > [linux/amd64 builder 3/3] RUN cd /go/src/github.com/openshift/eventrouter && go mod tidy && CGO_ENABLED=0 go build .:
0.179 go: downloading github.com/crewjam/rfc5424 v0.0.0-20180723152949-c25bdd3a0ba2
0.189 go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
0.240 go: downloading github.com/prometheus/client_golang v1.12.1
0.241 go: downloading k8s.io/api v0.27.1
0.242 go: downloading github.com/spf13/viper v1.4.0
0.559 go: downloading k8s.io/apimachinery v0.27.1
0.848 go: downloading k8s.io/client-go v0.27.1
0.925 go: downloading github.com/Shopify/sarama v1.23.1
0.927 go: downloading github.com/eapache/channels v1.1.0
0.960 go: downloading github.com/sethgrid/pester v0.0.0-20190127155807-68a33a018ad0
2.896 go: downloading github.com/prometheus/client_model v0.2.0
2.905 go: downloading github.com/prometheus/common v0.32.1
2.919 go: downloading github.com/beorn7/perks v1.0.1
2.922 go: downloading github.com/cespare/xxhash/v2 v2.1.2
2.957 go: downloading github.com/golang/protobuf v1.5.3
3.023 go: downloading github.com/prometheus/procfs v0.7.3
3.038 go: downloading golang.org/x/sys v0.6.0
3.096 go: downloading google.golang.org/protobuf v1.28.1
3.159 go: downloading github.com/fsnotify/fsnotify v1.4.9
3.166 go: downloading github.com/hashicorp/hcl v1.0.0
3.451 go: downloading github.com/magiconair/properties v1.8.0
3.499 go: downloading github.com/mitchellh/mapstructure v1.1.2
3.500 go: downloading github.com/pelletier/go-toml v1.2.0
3.557 go: downloading github.com/spf13/afero v1.2.2
3.578 go: downloading github.com/spf13/cast v1.3.0
3.616 go: downloading github.com/spf13/jwalterweatherman v1.0.0
3.634 go: downloading github.com/spf13/pflag v1.0.5
3.649 go: downloading gopkg.in/yaml.v2 v2.4.0
3.759 go: downloading github.com/eapache/queue v1.1.0
3.833 go: downloading k8s.io/klog/v2 v2.90.1
3.949 go: downloading github.com/DataDog/zstd v1.3.6-0.20190409195224-796139022798
4.124 go: downloading github.com/davecgh/go-spew v1.1.1
4.176 go: downloading github.com/eapache/go-resiliency v1.1.0
4.238 go: downloading github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21
4.275 go: downloading github.com/jcmturner/gofork v0.0.0-20190328161633-dc7c13fece03
4.321 go: downloading github.com/pierrec/lz4 v0.0.0-20190327172049-315a67e90e41
4.348 go: downloading github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a
4.396 go: downloading golang.org/x/net v0.8.0
4.435 go: downloading gopkg.in/jcmturner/gokrb5.v7 v7.2.3
9.354 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c
9.354 go: downloading github.com/stretchr/testify v1.8.1
23.32 go: downloading github.com/Shopify/toxiproxy v2.1.4+incompatible
23.32 go: downloading github.com/gogo/protobuf v1.3.2
23.32 go: downloading github.com/imdario/mergo v0.3.7
23.32 go: downloading golang.org/x/term v0.6.0
23.33 go: downloading sigs.k8s.io/yaml v1.3.0
23.39 go: downloading k8s.io/utils v0.0.0-20230209194617-a36077c30491
23.40 go: downloading github.com/google/go-cmp v0.5.9
23.72 go: downloading github.com/google/gofuzz v1.1.0
23.76 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
23.78 go: downloading github.com/BurntSushi/toml v0.3.1
23.79 go: downloading golang.org/x/text v0.8.0
23.80 go: downloading github.com/go-logr/logr v1.2.3
23.83 go: downloading github.com/golang/snappy v0.0.1
24.03 go: downloading github.com/kr/pretty v0.3.0
24.03 go: downloading github.com/pmezard/go-difflib v1.0.0
24.58 go: downloading gopkg.in/yaml.v3 v3.0.1
24.58 go: downloading github.com/hashicorp/go-uuid v1.0.1
24.71 go: downloading gopkg.in/jcmturner/dnsutils.v1 v1.0.1
24.84 go: downloading gopkg.in/jcmturner/goidentity.v3 v3.0.0
25.21 go: downloading gopkg.in/inf.v0 v0.9.1
25.29 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3
25.47 go: downloading github.com/google/gnostic v0.5.7-v3refs
25.49 go: downloading k8s.io/kube-openapi v0.0.0-20230308215209-15aac26d736a
25.69 go: downloading github.com/google/uuid v1.3.0
26.18 go: downloading golang.org/x/time v0.0.0-20220210224613-90d013bbcef8
26.18 go: downloading golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5
26.66 go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd
26.85 go: downloading github.com/kr/text v0.2.0
26.85 go: downloading github.com/rogpeppe/go-internal v1.10.0
26.89 go: downloading golang.org/x/crypto v0.0.0-20220315160706-3147a52a75dd
27.01 go: downloading gopkg.in/jcmturner/rpc.v1 v1.1.0
27.18 go: downloading github.com/json-iterator/go v1.1.12
27.59 go: downloading gopkg.in/jcmturner/aescts.v1 v1.0.1
27.59 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
27.61 go: downloading github.com/modern-go/reflect2 v1.0.2
27.62 go: downloading github.com/go-openapi/swag v0.22.3
27.64 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822
27.66 go: downloading google.golang.org/appengine v1.6.7
27.67 go: downloading github.com/onsi/ginkgo/v2 v2.9.1
27.89 go: downloading github.com/onsi/gomega v1.27.4
27.99 go: downloading github.com/go-openapi/jsonreference v0.20.1
27.99 go: downloading github.com/emicklei/go-restful/v3 v3.9.0
27.99 go: downloading github.com/mailru/easyjson v0.7.7
28.02 go: downloading github.com/go-openapi/jsonpointer v0.19.6
28.07 go: downloading github.com/josharian/intern v1.0.0
28.15 go: downloading github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0
28.15 go: downloading golang.org/x/tools v0.7.0
28.15 go: downloading github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1
48.30 runtime/metrics: /usr/local/go/pkg/tool/linux_amd64/compile: signal: segmentation fault (core dumped)
135.3 k8s.io/apimachinery/pkg/util/yaml: /usr/local/go/pkg/tool/linux_amd64/compile: signal: segmentation fault (core dumped)
------
Dockerfile:10
--------------------
   8 |         git clone https://github.com/openshift/eventrouter
   9 |     
  10 | >>> RUN cd /go/src/github.com/openshift/eventrouter && go mod tidy && CGO_ENABLED=0 go build .
  11 |     
  12 |     FROM alpine:3.14 as run
--------------------
ERROR: failed to solve: process "/dev/.buildkit_qemu_emulator /bin/sh -c cd /go/src/github.com/openshift/eventrouter && go mod tidy && CGO_ENABLED=0 go build ." did not complete successfully: exit code: 1
Additional info

This is the same symptom as issue #314 and my Dockerfile here is an updated version of @carlosedp's one in that issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Dockerfile's Go builder stage and reproduce the reported docker buildx build command, focusing on the linux/amd64 step that ends with a Go compiler segmentation fault. Compare its logs with the successful linux/arm64 and linux/ppc64le builds. Done means the reported multi-platform build completes without the amd64 compiler crash.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
build-system, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.