Unexpected CANCELED and CACHED operations with `COPY --link`
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- docker, go
- Domain
- build-system, devops
Research direction
Reproduce the report with the supplied Dockerfile and multi-platform docker buildx build command, then compare the progress output for COPY --link and COPY --from. Trace the BuildKit progress handling for the repeated CANCELED and CACHED operations; done means the output no longer reports misleading operations while the build still completes for all listed platforms.
Written by the indexing model from the issue text.
Description
BuildKit 0.10.6. I see a strange behavior in the progress output when COPY --link is used in the following Dockerfile:
# syntax=docker/dockerfile:1
ARG GO_VERSION="1.19"
ARG XX_VERSION="1.1.2"
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
FROM --platform=$BUILDPLATFORM golang:${GO_VERSION}-alpine AS base
COPY --link --from=xx / /
ENV CGO_ENABLED=0
RUN apk add --no-cache file git
WORKDIR /src
FROM base as build
RUN echo foo
ARG TARGETPLATFORM
RUN echo $TARGETPLATFORM
$ docker buildx create --use
$ docker buildx build --platform linux/amd64,linux/arm64,linux/s390x,linux/ppc64le --no-cache .
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 414B done
#1 DONE 0.1s
#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.1s
#3 resolve image config for docker.io/docker/dockerfile:1
#3 DONE 0.4s
#4 docker-image://docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc
#4 resolve docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc 0.0s done
#4 CACHED
#5 [linux/amd64 internal] load metadata for docker.io/library/golang:1.19-alpine
#5 DONE 1.1s
#6 [linux/amd64 internal] load metadata for docker.io/tonistiigi/xx:1.1.2
#6 DONE 1.2s
#7 [linux/amd64 xx 1/1] FROM docker.io/tonistiigi/xx:1.1.2@sha256:9dde7edeb9e4a957ce78be9f8c0fbabe0129bf5126933cd3574888f443731cda
#7 resolve docker.io/tonistiigi/xx:1.1.2@sha256:9dde7edeb9e4a957ce78be9f8c0fbabe0129bf5126933cd3574888f443731cda 0.0s done
#7 sha256:6b9f00cd577380cfa5d5a8d5ebf82f91f7aa469401818e4de53b2a16dd4a1642 0B / 14.97kB
#7 DONE 0.1s
#8 [linux/amd64 base 2/4] COPY --link --from=xx / /
#8 CANCELED
#9 [linux/amd64 base 2/4] COPY --link --from=xx / /
#9 CANCELED
#10 [linux/amd64 base 2/4] COPY --link --from=xx / /
#10 CANCELED
#11 [linux/amd64 base 1/4] FROM docker.io/library/golang:1.19-alpine@sha256:d171aa333fb386089206252503bc6ab545072670e0286e3d1bbc644362825c6e
#11 resolve docker.io/library/golang:1.19-alpine@sha256:d171aa333fb386089206252503bc6ab545072670e0286e3d1bbc644362825c6e 0.1s done
#11 DONE 0.3s
#7 [linux/amd64 xx 1/1] FROM docker.io/tonistiigi/xx:1.1.2@sha256:9dde7edeb9e4a957ce78be9f8c0fbabe0129bf5126933cd3574888f443731cda
#7 sha256:6b9f00cd577380cfa5d5a8d5ebf82f91f7aa469401818e4de53b2a16dd4a1642 14.97kB / 14.97kB 0.3s done
#7 extracting sha256:6b9f00cd577380cfa5d5a8d5ebf82f91f7aa469401818e4de53b2a16dd4a1642 0.0s done
#7 DONE 0.5s
#12 [linux/amd64 base 2/4] COPY --link --from=xx / /
#12 DONE 0.1s
#11 [linux/amd64 base 1/4] FROM docker.io/library/golang:1.19-alpine@sha256:d171aa333fb386089206252503bc6ab545072670e0286e3d1bbc644362825c6e
#11 sha256:c41ae7ad2b39813485dabd88ae2e5ddce71b71a47d19af1377b0251055edf373 0B / 284.75kB
#11 sha256:8cce5c603bfa087147c080777ac9cfd91f8c48a1a9dd2a8f416be618ca0b672a 0B / 156B
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 0B / 122.65MB
#11 sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce 0B / 2.81MB
#11 DONE 0.5s
#8 [linux/amd64 base 2/4] COPY --link --from=xx / /
#8 CANCELED
#13 [linux/amd64 base 3/4] RUN apk add --no-cache file git
#13 CANCELED
#9 [linux/amd64 base 2/4] COPY --link --from=xx / /
#9 CANCELED
#14 [linux/amd64 base 3/4] RUN apk add --no-cache file git
#14 CANCELED
#10 [linux/amd64 base 2/4] COPY --link --from=xx / /
#10 CANCELED
#15 [linux/amd64 base 3/4] RUN apk add --no-cache file git
#15 CANCELED
#11 [linux/amd64 base 1/4] FROM docker.io/library/golang:1.19-alpine@sha256:d171aa333fb386089206252503bc6ab545072670e0286e3d1bbc644362825c6e
#11 sha256:8cce5c603bfa087147c080777ac9cfd91f8c48a1a9dd2a8f416be618ca0b672a 156B / 156B 0.2s done
#11 sha256:c41ae7ad2b39813485dabd88ae2e5ddce71b71a47d19af1377b0251055edf373 284.75kB / 284.75kB 0.2s done
#11 sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce 1.05MB / 2.81MB 0.5s
#11 sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce 2.10MB / 2.81MB 0.6s
#11 ...
#12 [linux/amd64 base 2/4] COPY --link --from=xx / /
#12 extracting sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce 0.1s done
#12 DONE 0.2s
#11 [linux/amd64 base 1/4] FROM docker.io/library/golang:1.19-alpine@sha256:d171aa333fb386089206252503bc6ab545072670e0286e3d1bbc644362825c6e
#11 sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce 2.81MB / 2.81MB 0.7s done
#11 ...
#12 [linux/amd64 base 2/4] COPY --link --from=xx / /
#12 extracting sha256:c41ae7ad2b39813485dabd88ae2e5ddce71b71a47d19af1377b0251055edf373 0.1s done
#12 DONE 0.2s
#11 [linux/amd64 base 1/4] FROM docker.io/library/golang:1.19-alpine@sha256:d171aa333fb386089206252503bc6ab545072670e0286e3d1bbc644362825c6e
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 7.34MB / 122.65MB 1.1s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 14.68MB / 122.65MB 1.7s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 20.97MB / 122.65MB 2.1s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 27.26MB / 122.65MB 2.6s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 33.55MB / 122.65MB 3.0s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 42.99MB / 122.65MB 3.8s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 51.38MB / 122.65MB 4.4s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 58.72MB / 122.65MB 5.0s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 65.01MB / 122.65MB 5.4s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 72.35MB / 122.65MB 6.0s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 80.74MB / 122.65MB 6.6s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 87.03MB / 122.65MB 7.1s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 94.37MB / 122.65MB 7.7s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 100.66MB / 122.65MB 8.1s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 106.95MB / 122.65MB 8.6s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 113.25MB / 122.65MB 9.0s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 119.54MB / 122.65MB 9.5s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 122.65MB / 122.65MB 9.7s done
#11 DONE 10.2s
#12 [linux/amd64 base 2/4] COPY --link --from=xx / /
#12 extracting sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0
#12 extracting sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 2.1s done
#12 extracting sha256:8cce5c603bfa087147c080777ac9cfd91f8c48a1a9dd2a8f416be618ca0b672a 0.0s done
#12 merging 0.0s done
#12 DONE 2.4s
#16 [linux/amd64 base 3/4] RUN apk add --no-cache file git
#0 0.116 fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
#16 0.234 fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
#16 0.419 (1/8) Installing libmagic (5.41-r0)
#16 0.460 (2/8) Installing file (5.41-r0)
#16 0.469 (3/8) Installing brotli-libs (1.0.9-r6)
#16 0.498 (4/8) Installing nghttp2-libs (1.47.0-r0)
#16 0.513 (5/8) Installing libcurl (7.83.1-r4)
#16 0.543 (6/8) Installing expat (2.5.0-r0)
#16 0.569 (7/8) Installing pcre2 (10.40-r0)
#16 0.599 (8/8) Installing git (2.36.3-r0)
#16 1.008 Executing busybox-1.35.0-r17.trigger
#16 1.011 OK: 27 MiB in 23 packages
#16 DONE 1.2s
#17 [linux/amd64 base 4/4] WORKDIR /src
#17 DONE 0.1s
#18 [linux/amd64 build 1/2] RUN echo foo
#0 0.135 foo
#18 DONE 0.2s
#19 [linux/amd64 build 1/2] RUN echo foo
#19 CACHED
#20 [linux/amd64 base 4/4] WORKDIR /src
#20 CACHED
#21 [linux/amd64 base 4/4] WORKDIR /src
#21 CACHED
#22 [linux/amd64 build 1/2] RUN echo foo
#22 CACHED
#23 [linux/amd64 base 4/4] WORKDIR /src
#23 CACHED
#24 [linux/amd64 build 1/2] RUN echo foo
#24 CACHED
#25 [linux/amd64->arm64 build 2/2] RUN echo linux/arm64
#25 0.133 linux/arm64
#25 DONE 0.2s
#26 [linux/amd64->ppc64le build 2/2] RUN echo linux/ppc64le
#0 0.147 linux/ppc64le
#26 DONE 0.2s
#27 [linux/amd64 build 2/2] RUN echo linux/amd64
#0 0.153 linux/amd64
#27 DONE 0.2s
#28 [linux/amd64->s390x build 2/2] RUN echo linux/s390x
#0 0.162 linux/s390x
#28 DONE 0.2s
Without COPY --link:
# syntax=docker/dockerfile:1
ARG GO_VERSION="1.19"
ARG XX_VERSION="1.1.2"
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
FROM --platform=$BUILDPLATFORM golang:${GO_VERSION}-alpine AS base
COPY --from=xx / /
ENV CGO_ENABLED=0
RUN apk add --no-cache file git
WORKDIR /src
FROM base as build
RUN echo foo
ARG TARGETPLATFORM
RUN echo $TARGETPLATFORM
$ docker buildx build --platform linux/amd64,linux/arm64,linux/s390x,linux/ppc64le --no-cache .
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 407B done
#1 DONE 0.1s
#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.1s
#3 [auth] docker/dockerfile:pull token for registry-1.docker.io
#3 DONE 0.0s
#4 resolve image config for docker.io/docker/dockerfile:1
#4 DONE 0.7s
#5 docker-image://docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc
#5 resolve docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc 0.0s done
#5 CACHED
#6 [auth] library/golang:pull token for registry-1.docker.io
#6 DONE 0.0s
#7 [auth] tonistiigi/xx:pull token for registry-1.docker.io
#7 DONE 0.0s
#8 [linux/amd64 internal] load metadata for docker.io/library/golang:1.19-alpine
#8 DONE 1.2s
#9 [linux/amd64 internal] load metadata for docker.io/tonistiigi/xx:1.1.2
#9 DONE 1.5s
#10 [linux/amd64 xx 1/1] FROM docker.io/tonistiigi/xx:1.1.2@sha256:9dde7edeb9e4a957ce78be9f8c0fbabe0129bf5126933cd3574888f443731cda
#10 resolve docker.io/tonistiigi/xx:1.1.2@sha256:9dde7edeb9e4a957ce78be9f8c0fbabe0129bf5126933cd3574888f443731cda 0.0s done
#10 ...
#11 [linux/amd64 base 1/4] FROM docker.io/library/golang:1.19-alpine@sha256:d171aa333fb386089206252503bc6ab545072670e0286e3d1bbc644362825c6e
#11 resolve docker.io/library/golang:1.19-alpine@sha256:d171aa333fb386089206252503bc6ab545072670e0286e3d1bbc644362825c6e 0.0s done
#11 DONE 0.3s
#10 [linux/amd64 xx 1/1] FROM docker.io/tonistiigi/xx:1.1.2@sha256:9dde7edeb9e4a957ce78be9f8c0fbabe0129bf5126933cd3574888f443731cda
#10 sha256:6b9f00cd577380cfa5d5a8d5ebf82f91f7aa469401818e4de53b2a16dd4a1642 0B / 14.97kB 0.2s
#10 sha256:6b9f00cd577380cfa5d5a8d5ebf82f91f7aa469401818e4de53b2a16dd4a1642 14.97kB / 14.97kB 0.4s done
#10 extracting sha256:6b9f00cd577380cfa5d5a8d5ebf82f91f7aa469401818e4de53b2a16dd4a1642 0.0s done
#10 DONE 0.6s
#11 [linux/amd64 base 1/4] FROM docker.io/library/golang:1.19-alpine@sha256:d171aa333fb386089206252503bc6ab545072670e0286e3d1bbc644362825c6e
#11 sha256:8cce5c603bfa087147c080777ac9cfd91f8c48a1a9dd2a8f416be618ca0b672a 156B / 156B 0.3s done
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 0B / 122.65MB 0.3s
#11 sha256:c41ae7ad2b39813485dabd88ae2e5ddce71b71a47d19af1377b0251055edf373 284.75kB / 284.75kB 0.2s done
#11 sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce 0B / 2.81MB 0.2s
#11 sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce 1.05MB / 2.81MB 0.5s
#11 sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce 2.81MB / 2.81MB 0.7s done
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 6.29MB / 122.65MB 1.1s
#11 extracting sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce 0.1s done
#11 extracting sha256:c41ae7ad2b39813485dabd88ae2e5ddce71b71a47d19af1377b0251055edf373 0.1s done
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 14.68MB / 122.65MB 1.7s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 20.97MB / 122.65MB 2.1s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 28.31MB / 122.65MB 2.7s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 36.70MB / 122.65MB 3.3s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 46.14MB / 122.65MB 4.1s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 53.48MB / 122.65MB 4.7s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 59.77MB / 122.65MB 5.1s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 66.06MB / 122.65MB 5.6s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 73.40MB / 122.65MB 6.2s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 79.69MB / 122.65MB 6.6s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 85.98MB / 122.65MB 7.1s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 93.32MB / 122.65MB 7.7s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 99.61MB / 122.65MB 8.1s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 106.95MB / 122.65MB 8.7s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 115.34MB / 122.65MB 9.3s
#11 sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 122.65MB / 122.65MB 9.8s done
#11 extracting sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0
#11 extracting sha256:15b491bbc2fee14d5914da432332ef2992302d2895ae90466c778c259e643fe0 1.9s done
#11 extracting sha256:8cce5c603bfa087147c080777ac9cfd91f8c48a1a9dd2a8f416be618ca0b672a 0.0s done
#11 DONE 12.1s
#12 [linux/amd64 base 2/4] COPY --from=xx / /
#12 DONE 0.2s
#13 [linux/amd64 base 3/4] RUN apk add --no-cache file git
#13 0.147 fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
#13 0.289 fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
#13 0.492 (1/8) Installing libmagic (5.41-r0)
#13 0.538 (2/8) Installing file (5.41-r0)
#13 0.547 (3/8) Installing brotli-libs (1.0.9-r6)
#13 0.576 (4/8) Installing nghttp2-libs (1.47.0-r0)
#13 0.590 (5/8) Installing libcurl (7.83.1-r4)
#13 0.613 (6/8) Installing expat (2.5.0-r0)
#13 0.627 (7/8) Installing pcre2 (10.40-r0)
#13 0.682 (8/8) Installing git (2.36.3-r0)
#13 1.100 Executing busybox-1.35.0-r17.trigger
#13 1.103 OK: 27 MiB in 23 packages
#13 DONE 1.2s
#14 [linux/amd64 base 4/4] WORKDIR /src
#14 DONE 0.1s
#15 [linux/amd64 build 1/2] RUN echo foo
#15 0.082 foo
#15 DONE 0.1s
#16 [linux/amd64->s390x build 2/2] RUN echo linux/s390x
#16 0.166 linux/s390x
#16 DONE 0.2s
#17 [linux/amd64->ppc64le build 2/2] RUN echo linux/ppc64le
#0 0.125 linux/ppc64le
#17 DONE 0.2s
#18 [linux/amd64->arm64 build 2/2] RUN echo linux/arm64
#0 0.145 linux/arm64
#18 DONE 0.2s
#19 [linux/amd64 build 2/2] RUN echo linux/amd64
#0 0.154 linux/amd64
#19 DONE 0.2s
- Dominant language
- Go
- Stars
- 10.3k
- Forks
- 1.5k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 48
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from moby/buildkit
-
status/triage
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
area/dockerfile
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Similar issues
-
optimization optimization:agents-md-curator
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
githubnext/gh-aw-cao#13143 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
blinklabs-io/bursa#904 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
yanet-platform/ipfw-go#129 ·
-
bug confmap/provider/googlesecretmanagerprovider needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
open-telemetry/opentelemetry-collector-contrib#51273 · 2 comments ·
-
bug: AI Gateway client filter lists "Unknown" twice when NULL and literal Unknown clients coexist Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100