docker / docker/buildx

`--cache-from` takes longer than actual (cached) build

Open
#3,491 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/cache kind/enhancement
Dominant language
Go
Stars
4.5k
Forks
682
Avg merge
2d 14h
Merged PRs (30d)
29

Description

Contributing guidelines
I've found a bug and checked that ...
  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem
Description

When re-building a (large) multi staged Dockerfile, most time is spent in the --cache-from=type=registry step "importing cache manifest from docker.io/fenollexai/supergreen" while the (cached) build is actually over and done.

Expected behaviour

I'd expect that cache import be cancelled as soon as build is done. Especially since --cache-from cannot produce build errors.

Actual behaviour

Instead, my finished build is left hanging on the import step.

Buildx version

github.com/docker/buildx v0.29.1-desktop.1 28f6246ff24e2c05095e8741e48c48dcb2d3b4bc

Docker info
Client:
 Version:    28.5.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Docker AI Agent - Ask Gordon (Docker Inc.)
    Version:  v1.9.11
    Path:     /Users/pierre/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.29.1-desktop.1
    Path:     /Users/pierre/.docker/cli-plugins/docker-buildx
  cloud: Docker Cloud (Docker Inc.)
    Version:  v0.4.39
    Path:     /Users/pierre/.docker/cli-plugins/docker-cloud
  compose: Docker Compose (Docker Inc.)
    Version:  v2.40.2-desktop.1
    Path:     /Users/pierre/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.44
    Path:     /Users/pierre/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Docker Inc.)
    Version:  v0.2.0
    Path:     /Users/pierre/.docker/cli-plugins/docker-desktop
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.31
    Path:     /Users/pierre/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/pierre/.docker/cli-plugins/docker-init
  mcp: Docker MCP Plugin (Docker Inc.)
    Version:  v0.24.0
    Path:     /Users/pierre/.docker/cli-plugins/docker-mcp
  model: Docker Model Runner (Docker Inc.)
    Version:  v0.1.42
    Path:     /Users/pierre/.docker/cli-plugins/docker-model
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/pierre/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.18.3
    Path:     /Users/pierre/.docker/cli-plugins/docker-scout

Server:
 Containers: 3
  Running: 1
  Paused: 0
  Stopped: 2
 Images: 27
 Server Version: 28.5.1
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Discovered Devices:
  cdi: docker.com/gpu=webgpu
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.10.14-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 7.654GiB
 Name: docker-desktop
 ID: d987371f-97b4-4975-960d-8d656a6e354d
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Username: fenollexai
 Labels:
  com.docker.desktop.address=unix:///Users/pierre/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
Builders list
NAME/NODE           DRIVER/ENDPOINT     STATUS    BUILDKIT   PLATFORMS
supergreen          docker-container                         
 \_ supergreen0      \_ desktop-linux   running   v0.25.1    linux/amd64 (+2), linux/arm64, linux/arm (+2), linux/ppc64le, (3 more)
default             docker                                   
 \_ default          \_ default         running   v0.25.1    linux/amd64 (+2), linux/arm64, linux/ppc64le, linux/s390x, (2 more)
desktop-linux*      docker                                   
 \_ desktop-linux    \_ desktop-linux   running   v0.25.1    linux/amd64 (+2), linux/arm64, linux/ppc64le, linux/s390x, (2 more)
Configuration
BUILDX_BUILDER="supergreen" docker build --output=type=cacheonly --cache-from=type=registry,ref=docker.io/fenollexai/supergreen - </var/folders/c3/n7r_gdc56rb9fmdly4p20lk00000gn/T/cargo-green-fetch-692e2e6.Dockerfile

cargo-green-fetch-692e2e6_Dockerfile.txt

Build logs
[+] Building 3.5s (529/529) FINISHED                                                                                                                                                                                  docker-container:supergreen
 => [internal] load build definition from Dockerfile                                                                                                                                                                                         0.0s
 => => transferring dockerfile: 49.10kB                                                                                                                                                                                                      0.0s
 => resolve image config for docker-image://docker.io/docker/dockerfile:1@sha256:b6afd42430b15f2d2a4c5a02b919e98a525b785b1aaff16747d2f623364e39b6                                                                                            0.3s
 => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:b6afd42430b15f2d2a4c5a02b919e98a525b785b1aaff16747d2f623364e39b6                                                                                                              0.0s
 => => resolve docker.io/docker/dockerfile:1@sha256:b6afd42430b15f2d2a4c5a02b919e98a525b785b1aaff16747d2f623364e39b6                                                                                                                         0.0s
 => [internal] load metadata for docker.io/library/rust:1.90.0-slim                                                                                                                                                                          0.6s
 => [internal] load metadata for docker.io/moby/buildkit:latest                                                                                                                                                                              0.5s
 => [internal] load metadata for docker.io/docker/dockerfile:1                                                                                                                                                                               0.4s
 => [internal] load .dockerignore                                                                                                                                                                                                            0.0s
 => => transferring context: 2B                                                                                                                                                                                                              0.0s
 => importing cache manifest from docker.io/fenollexai/supergreen                                                                                                                                                                            2.1s
 => => inferred cache manifest type: application/vnd.oci.image.manifest.v1+json                                                                                                                                                              0.0s
 => [docker.io-moby-buildkit-latest 1/1] FROM docker.io/moby/buildkit:latest@sha256:79cc6476ab1a3371c9afd8b44e7c55610057c43e18d9b39b68e2b0c2475cc1b6                                                                                         0.0s
 => => resolve docker.io/moby/buildkit:latest@sha256:79cc6476ab1a3371c9afd8b44e7c55610057c43e18d9b39b68e2b0c2475cc1b6                                                                                                                        0.0s
 => [cargo-fetch-1 119/127] ADD --chmod=0664 --unpack=true --checksum=sha256:cfe53a6657fd280eaa890a3bc59152892ffa3e30101319d168b781ed6529b049   https://static.crates.io/crates/yansi/yansi-1.0.1.crate /                                    0.0s
 => [cargo-fetch-1 125/127] ADD --chmod=0664 --unpack=true --checksum=sha256:d71e5d6e06ab090c67b5e44993ec16b72dcbaabc526db883a360057678b48502   https://static.crates.io/crates/zerofrom-derive/zerofrom-derive-0.1.6.crate /                0.0s
 => [cargo-fetch-0 120/127] ADD --chmod=0664 --unpack=true --checksum=sha256:d8a2f0d8d040d7848a709caf78912debcc3f33ee4b3cac47d73d1e1069e83507   https://static.crates.io/crates/portable-atomic-util/portable-atomic-util-0.2.4.crate /      0.0s
 => [cargo-fetch-0 125/127] ADD --chmod=0664 --unpack=true --checksum=sha256:626214629cda6781b6dc1d316ba307189c85ba657213ce642d9c77670f8202c8   https://static.crates.io/crates/quinn/quinn-0.11.8.crate /                                   0.0s
 => [cargo-fetch-1 124/127] ADD --chmod=0664 --unpack=true --checksum=sha256:50cc42e0333e05660c3587f3bf9d0478688e15d870fab3346451ce7f8c9fbea5   https://static.crates.io/crates/zerofrom/zerofrom-0.1.6.crate /                              0.0s
 => [cargo-fetch-1 118/127] ADD --chmod=0664 --unpack=true --checksum=sha256:ea2f10b9bb0928dfb1b42b65e1f9e36f7f54dbdf08457afefb38afcdec4fa2bb   https://static.crates.io/crates/writeable/writeable-0.6.1.crate /                            0.0s
 => [cargo-fetch-0 124/127] ADD --chmod=0664 --unpack=true --checksum=sha256:89ae43fd86e4158d6db51ad8e2b80f313af9cc74f5c0e03ccb87de09998732de   https://static.crates.io/crates/proc-macro2/proc-macro2-1.0.101.crate /                      0.0s
 => [cargo-fetch-0 118/127] ADD --chmod=0664 --unpack=true --checksum=sha256:8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184   https://static.crates.io/crates/pin-utils/pin-utils-0.1.0.crate /                            0.0s
 => [cargo-fetch-1 123/127] ADD --chmod=0664 --unpack=true --checksum=sha256:9ecf5b4cc5364572d7f4c329661bcc82724222973f2cab6f050a4e5c22f75181   https://static.crates.io/crates/zerocopy-derive/zerocopy-derive-0.8.26.crate /               0.0s
 => [cargo-fetch-0 123/127] ADD --chmod=0664 --unpack=true --checksum=sha256:3ae130e2f271fbc2ac3a40fb1d07180839cdbbe443c7a27e1e3c13c5cac0116d   https://static.crates.io/crates/pretty_assertions/pretty_assertions-1.4.1.crate /            0.0s
 => [cargo-fetch-0 119/127] ADD --chmod=0664 --unpack=true --checksum=sha256:f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483   https://static.crates.io/crates/portable-atomic/portable-atomic-1.11.1.crate /               0.0s
 => [cargo-fetch-1 122/127] ADD --chmod=0664 --unpack=true --checksum=sha256:1039dd0d3c310cf05de012d8a39ff557cb0d23087fd44cad61df08fc31907a2f   https://static.crates.io/crates/zerocopy/zerocopy-0.8.26.crate /                             0.0s
 => [cargo-fetch-0 122/127] ADD --chmod=0664 --unpack=true --checksum=sha256:85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9   https://static.crates.io/crates/ppv-lite86/ppv-lite86-0.2.21.crate /                         0.0s
 => [docker.io-library-rust-1.90.0-slim 1/1] FROM docker.io/library/rust:1.90.0-slim@sha256:7fa728f3678acf5980d5db70960cf8491aff9411976789086676bdf0c19db39e                                                                                 0.0s
 => => resolve docker.io/library/rust:1.90.0-slim@sha256:7fa728f3678acf5980d5db70960cf8491aff9411976789086676bdf0c19db39e                                                                                                                    0.0s
 => [cargo-fetch-1 121/127] ADD --chmod=0664 --unpack=true --checksum=sha256:38da3c9736e16c5d3c8c597a9aaa5d1fa565d0532ae05e27c24aa62fb32c0ab6   https://static.crates.io/crates/yoke-derive/yoke-derive-0.8.0.crate /                        0.0s
 => [cargo-fetch-0 121/127] ADD --chmod=0664 --unpack=true --checksum=sha256:e5a7c30837279ca13e7c867e9e40053bc68740f988cb07f7ca6df43cc734b585   https://static.crates.io/crates/potential_utf/potential_utf-0.1.2.crate /                    0.0s
 => [cargo-fetch-1 120/127] ADD --chmod=0664 --unpack=true --checksum=sha256:5f41bb01b8226ef4bfd589436a297c53d118f65921786300e427be8d487695cc   https://static.crates.io/crates/yoke/yoke-0.8.0.crate /                                      0.0s
 => [docker.io-docker-dockerfile-1 1/1] FROM docker.io/docker/dockerfile:1@sha256:b6afd42430b15f2d2a4c5a02b919e98a525b785b1aaff16747d2f623364e39b6                                                                                           0.0s
 => => resolve docker.io/docker/dockerfile:1@sha256:b6afd42430b15f2d2a4c5a02b919e98a525b785b1aaff16747d2f623364e39b6                                                                                                                         0.0s
 => [cargo-fetch-2 2/2] ADD --chmod=0664 --unpack=true --checksum=sha256:5b96237efa0c878c64bd89c436f661be4e46b2f3eff1ebb976f7ef2321d2f58f   https://static.crates.io/crates/zerovec-derive/zerovec-derive-0.11.1.crate /                     0.0s
 => [cargo-fetch-1 127/127] ADD --chmod=0664 --unpack=true --checksum=sha256:36f0bbd478583f79edad978b407914f61b2972f5af6fa089686016be8f9af595   https://static.crates.io/crates/zerotrie/zerotrie-0.2.2.crate /                              0.0s
 => [cargo-fetch-2 1/2] ADD --chmod=0664 --unpack=true --checksum=sha256:e7aa2bd55086f1ab526693ecbe444205da57e25f4489879da80635a46d90e73b   https://static.crates.io/crates/zerovec/zerovec-0.11.4.crate /                                   0.0s
 => [cargo-fetch-0 127/127] ADD --chmod=0664 --unpack=true --checksum=sha256:fcebb1209ee276352ef14ff8732e24cc2b02bbac986cd74a4c81bcb2f9881970   https://static.crates.io/crates/quinn-udp/quinn-udp-0.5.13.crate /                           0.0s
 => [cargo-fetch-1 126/127] ADD --chmod=0664 --unpack=true --checksum=sha256:ced3678a2879b30306d323f4542626697a464a97c0a07c9aebf7ebca65cd4dde   https://static.crates.io/crates/zeroize/zeroize-1.8.1.crate /                                0.0s
 => [cargo-fetch-0 126/127] ADD --chmod=0664 --unpack=true --checksum=sha256:49df843a9161c85bb8aae55f101bc0bac8bcafd637a620d9122fd7e0b2f7422e   https://static.crates.io/crates/quinn-proto/quinn-proto-0.11.12.crate /                      0.0s
 => [cargo-fetch-1 117/127] ADD --chmod=0664 --unpack=true --checksum=sha256:6f42320e61fe2cfd34354ecb597f86f413484a798ba44a8ca1165c58d42da6c1   https://static.crates.io/crates/wit-bindgen-rt/wit-bindgen-rt-0.39.0.crate /                 0.0s
 => [cargo-fetch-0 117/127] ADD --chmod=0664 --unpack=true --checksum=sha256:3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b   https://static.crates.io/crates/pin-project-lite/pin-project-lite-0.2.16.crate /             0.0s
 => [cargo-fetch-1 116/127] ADD --chmod=0664 --unpack=true --checksum=sha256:d135d17ab770252ad95e9a872d365cf3090e3be864a34ab46f48555993efc904   https://static.crates.io/crates/winsafe/winsafe-0.0.19.crate /                               0.0s
 => [cargo-fetch-0 116/127] ADD --chmod=0664 --unpack=true --checksum=sha256:5be167a7af36ee22fe3115051bc51f6e6c7054c9348e28deb4f49bd6f705a315   https://static.crates.io/crates/pico-args/pico-args-0.5.0.crate /                            0.0s
 => [cargo-fetch-1 115/127] ADD --chmod=0664 --unpack=true --checksum=sha256:21a0236b59786fed61e2a80582dd500fe61f18b5dca67a4a067d0bc9039339cf   https://static.crates.io/crates/winnow/winnow-0.7.13.crate /                                 0.0s
 => [cargo-fetch-0 115/127] ADD --chmod=0664 --unpack=true --checksum=sha256:e3148f5046208a5d56bcfc03053e3ca6334e51da8dfb19b6cdc8b306fae3283e   https://static.crates.io/crates/percent-encoding/percent-encoding-2.3.1.crate /              0.0s
 
truncated because of github form

  => CACHED [cargo-fetch-1 123/127] ADD --chmod=0664 --unpack=true --checksum=sha256:9ecf5b4cc5364572d7f4c329661bcc82724222973f2cab6f050a4e5c22f75181   https://static.crates.io/crates/zerocopy-derive/zerocopy-derive-0.8.26.crate /        0.0s
 => CACHED [cargo-fetch-1 124/127] ADD --chmod=0664 --unpack=true --checksum=sha256:50cc42e0333e05660c3587f3bf9d0478688e15d870fab3346451ce7f8c9fbea5   https://static.crates.io/crates/zerofrom/zerofrom-0.1.6.crate /                       0.0s
 => CACHED [cargo-fetch-1 125/127] ADD --chmod=0664 --unpack=true --checksum=sha256:d71e5d6e06ab090c67b5e44993ec16b72dcbaabc526db883a360057678b48502   https://static.crates.io/crates/zerofrom-derive/zerofrom-derive-0.1.6.crate /         0.0s
 => CACHED [cargo-fetch-1 126/127] ADD --chmod=0664 --unpack=true --checksum=sha256:ced3678a2879b30306d323f4542626697a464a97c0a07c9aebf7ebca65cd4dde   https://static.crates.io/crates/zeroize/zeroize-1.8.1.crate /                         0.0s
 => CACHED [cargo-fetch-1 127/127] ADD --chmod=0664 --unpack=true --checksum=sha256:36f0bbd478583f79edad978b407914f61b2972f5af6fa089686016be8f9af595   https://static.crates.io/crates/zerotrie/zerotrie-0.2.2.crate /                       0.0s
 => CACHED [cargo-fetch 2/6] COPY --from=cargo-fetch-1 / /                                                                                                                                                                                   0.0s
 => CACHED [cargo-fetch-2 1/2] ADD --chmod=0664 --unpack=true --checksum=sha256:e7aa2bd55086f1ab526693ecbe444205da57e25f4489879da80635a46d90e73b   https://static.crates.io/crates/zerovec/zerovec-0.11.4.crate /                            0.0s
 => CACHED [cargo-fetch-2 2/2] ADD --chmod=0664 --unpack=true --checksum=sha256:5b96237efa0c878c64bd89c436f661be4e46b2f3eff1ebb976f7ef2321d2f58f   https://static.crates.io/crates/zerovec-derive/zerovec-derive-0.11.1.crate /              0.0s
 => CACHED [cargo-fetch 3/6] COPY --from=cargo-fetch-2 / /                                                                                                                                                                                   0.0s
 => CACHED [cargo-fetch 4/6] COPY --from=docker.io-docker-dockerfile-1 / /docker.io-docker-dockerfile-1                                                                                                                                      0.0s
 => CACHED [cargo-fetch 5/6] COPY --from=docker.io-library-rust-1.90.0-slim / /docker.io-library-rust-1.90.0-slim                                                                                                                            0.0s
 => CACHED [cargo-fetch 6/6] COPY --from=docker.io-moby-buildkit-latest / /docker.io-moby-buildkit-latest                                                                                                                                    0.0s
Additional info

supergreen

  • uses Driver Options: image="docker.io/moby/buildkit:latest@sha256:79cc6476ab1a3371c9afd8b44e7c55610057c43e18d9b39b68e2b0c2475cc1b6"
  • also:
# docker logs 83725fd9ecec |& ag 'auto snapshotter'
time="2025-10-20T18:36:18Z" level=info msg="auto snapshotter: using overlayfs"

This is probably a buildkit issue however I am sorry I didn't take the time to turn this reproducer into a buildkit one.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the delay with the supplied cargo-green-fetch-692e2e6.Dockerfile and the documented BUILDX_BUILDER command, then inspect the cache-manifest import step shown in the build logs. Done means a cached build no longer waits on cache import after the build has completed, while preserving the reported build result.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
cli, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.