docker / docker/buildx

Cache miss when pushing only 1 target

Open
#576 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

area/bake area/cache
Dominant language
Go
Stars
4.5k
Forks
682
Avg merge
2d 14h
Merged PRs (30d)
29

Description

There seems to be a cache miss when you build all targets but only push one target to a repository. Please find the relevant logs and config below.

I am having the same issue. Two consecutive runs do have a cache miss

Docker version

ubuntu@ip-172-31-88-222:~$ docker --version
Docker version 20.10.5, build 55c4c88

bake.hcl

variable "CT_ECR_PREFIX" {
  default = ""
}

variable "CT_DOCKER_CACHE" {
  default = ""
}

variable "GITHUB_SHA" {
}

variable "DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME" {
}

variable "CT_ECR_RELEASE_TAG" {
}

group "default" {
	targets = ["app", "fake_aws"]
}

target "cache" {
  cache-to = ["type=inline,mode=max"]
  cache-from = [
    "${CT_ECR_PREFIX}ct-base:${DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME}",
    "${CT_ECR_PREFIX}ct-base:latest",
    "${CT_ECR_PREFIX}ct-base:ct-addon_${DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME}",
    "${CT_ECR_PREFIX}ct-base:ct-addon_latest"
  ]
}

target "app" {
	inherits = ["cache"]
  args = {
    CT_DOCKER_CACHE = "${CT_DOCKER_CACHE}"
    CT_ECR_PREFIX = "${CT_ECR_PREFIX}"
    GITHUB_SHA = "${GITHUB_SHA}"
  }
  tags = [
    "${CT_ECR_PREFIX}ct-base:ct-addon_${GITHUB_SHA}",
    "${CT_ECR_PREFIX}ct-base:ct-addon_${DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME}",
    "${CT_ECR_PREFIX}ct-base:ct-addon_${CT_ECR_RELEASE_TAG}"
  ]
}

target "fake_aws" {
	inherits = ["cache"]
  context = "../../../tools/mock_servers/fake_aws"
  args = {
    CT_DOCKER_CACHE = "${CT_DOCKER_CACHE}"
    CT_ECR_PREFIX = "${CT_ECR_PREFIX}"
    GITHUB_SHA = "${GITHUB_SHA}"
  }
  tags = [
    "ct-fake-aws:${GITHUB_SHA}",
    "ct-fake-aws:${DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME}",
    "ct-fake-aws:${CT_ECR_RELEASE_TAG}"
  ]
}

Build without push

ubuntu@ip-172-31-88-222:~/workspace/foo_ct-backend_PR-10594/js/service/addon$ CI=true CT_DOCKER_CACHE=harbor.infra.foo.com/cache/ CT_ECR_PREFIX=01234567890.dkr.ecr.eu-west-1.amazonaws.com/ DOCKER_CLI_EXPERIMENTAL=enabled GITHUB_SHA=a4e979fac131a5b8c39d4e2f0e8b4f745730941f DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME=chore_core-buildkit CT_ECR_RELEASE_TAG=chore_core-buildkit docker buildx bake -f bake.hcl
[+] Building 3.5s (21/21) FINISHED
 => [app internal] load build definition from Dockerfile                                                                                                                                                                                                                   0.0s
 => => transferring dockerfile: 32B                                                                                                                                                                                                                                        0.0s
 => [fake_aws internal] load build definition from Dockerfile                                                                                                                                                                                                              0.1s
 => => transferring dockerfile: 243B                                                                                                                                                                                                                                       0.0s
 => [app internal] load .dockerignore                                                                                                                                                                                                                                      0.1s
 => => transferring context: 35B                                                                                                                                                                                                                                           0.0s
 => [fake_aws internal] load .dockerignore                                                                                                                                                                                                                                 0.1s
 => => transferring context: 109B                                                                                                                                                                                                                                          0.0s
 => [fake_aws internal] load metadata for 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:a4e979fac131a5b8c39d4e2f0e8b4f745730941f                                                                                                                                    0.0s
 => [auth] sharing credentials for 01234567890.dkr.ecr.eu-west-1.amazonaws.com                                                                                                                                                                                            0.0s
 => [fake_aws] importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:chore_core-buildkit                                                                                                                                                      0.0s
 => [fake_aws] importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:latest                                                                                                                                                                   0.2s
 => [fake_aws] importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit                                                                                                                                             0.0s
 => [app] importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_latest                                                                                                                                                               3.2s
 => [fake_aws 1/4] FROM 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:a4e979fac131a5b8c39d4e2f0e8b4f745730941f                                                                                                                                                      0.0s
 => [app internal] load build context                                                                                                                                                                                                                                      0.0s
 => => transferring context: 10.16kB                                                                                                                                                                                                                                       0.0s
 => [auth] sharing credentials for 01234567890.dkr.ecr.eu-west-1.amazonaws.com                                                                                                                                                                                            0.0s
 => [fake_aws internal] load build context                                                                                                                                                                                                                                 0.1s
 => => transferring context: 4.61kB                                                                                                                                                                                                                                        0.0s
 => CACHED [app 2/4] WORKDIR /home/ct/js/service/addon                                                                                                                                                                                                                     0.0s
 => CACHED [app 3/4] COPY . .                                                                                                                                                                                                                                              0.0s
 => CACHED [app 4/4] RUN  yarn build                                                                                                                                                                                                                                       0.0s
 => CACHED [fake_aws 2/3] WORKDIR /home/ct/tools/mock_servers/fake_aws                                                                                                                                                                                                     0.0s
 => CACHED [fake_aws 3/3] COPY . /home/ct/tools/mock_servers/fake_aws                                                                                                                                                                                                      0.0s
 => [fake_aws] exporting to image                                                                                                                                                                                                                                          0.1s
 => => exporting layers                                                                                                                                                                                                                                                    0.0s
 => => writing image sha256:b762d9b76d272ac22305cbcbefef2a28ecb6d70baffecd008e75913d0620ddc8                                                                                                                                                                               0.0s
 => => naming to 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f                                                                                                                                                    0.0s
 => => naming to 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit                                                                                                                                                                         0.0s
 => => writing image sha256:d770cc800c787c7823debc6249cf26904b26c0e3f90ae00f7121c495ae909918                                                                                                                                                                               0.0s
 => => naming to docker.io/library/ct-fake-aws:a4e979fac131a5b8c39d4e2f0e8b4f745730941f                                                                                                                                                                                    0.0s
 => => naming to docker.io/library/ct-fake-aws:chore_core-buildkit                                                                                                                                                                                                         0.0s
 => [fake_aws] exporting cache                                                                                                                                                                                                                                             0.0s
 => => preparing build cache for export                                                                                                                                                                                                                                    0.0s

Docker image hash

ubuntu@ip-172-31-88-222:~$ docker inspect 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f | jq -r '.[].Id'
sha256:8fbb5dd0bff70b92372826d966cfb7c3452cfb89bccd2a9b484b2e4bfabc0851

Build with push

ubuntu@ip-172-31-88-222:~/workspace/foo_ct-backend_PR-10594/js/service/addon$ CI=true CT_DOCKER_CACHE=harbor.infra.foo.com/cache/ CT_ECR_PREFIX=01234567890.dkr.ecr.eu-west-1.amazonaws.com/ DOCKER_CLI_EXPERIMENTAL=enabled GITHUB_SHA=a4e979fac131a5b8c39d4e2f0e8b4f745730941f DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME=chore_core-buildkit CT_ECR_RELEASE_TAG=chore_core-buildkit docker buildx bake -f bake.hcl --push app
[+] Building 1.2s (17/17) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                       0.0s
 => => transferring dockerfile: 208B                                                                                                                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                          0.0s
 => => transferring context: 268B                                                                                                                                                                                                                                          0.0s
 => [internal] load metadata for 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:a4e979fac131a5b8c39d4e2f0e8b4f745730941f                                                                                                                                             0.0s
 => importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:latest                                                                                                                                                                              0.2s
 => importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit                                                                                                                                                        0.0s
 => importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:chore_core-buildkit                                                                                                                                                                 0.0s
 => importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_latest                                                                                                                                                                     0.2s
 => [1/4] FROM 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:a4e979fac131a5b8c39d4e2f0e8b4f745730941f                                                                                                                                                               0.0s
 => [internal] load build context                                                                                                                                                                                                                                          0.1s
 => => transferring context: 8.92MB                                                                                                                                                                                                                                        0.1s
 => [auth] sharing credentials for 01234567890.dkr.ecr.eu-west-1.amazonaws.com                                                                                                                                                                                            0.0s
 => CACHED [2/4] WORKDIR /home/ct/js/service/addon                                                                                                                                                                                                                         0.0s
 => CACHED [3/4] COPY . .                                                                                                                                                                                                                                                  0.0s
 => CACHED [4/4] RUN  yarn build                                                                                                                                                                                                                                           0.0s
 => exporting to image                                                                                                                                                                                                                                                     0.1s
 => => exporting layers                                                                                                                                                                                                                                                    0.0s
 => => writing image sha256:b762d9b76d272ac22305cbcbefef2a28ecb6d70baffecd008e75913d0620ddc8                                                                                                                                                                               0.0s
 => => naming to 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f                                                                                                                                                    0.0s
 => => naming to 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit                                                                                                                                                                         0.0s
 => exporting cache                                                                                                                                                                                                                                                        0.0s
 => => preparing build cache for export                                                                                                                                                                                                                                    0.0s
 => pushing 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f with docker                                                                                                                                             0.4s
 => => pushing layer e78ea794e900                                                                                                                                                                                                                                          0.5s
 => => pushing layer 04203156884c                                                                                                                                                                                                                                          0.5s
 => => pushing layer 5f70bf18a086                                                                                                                                                                                                                                          0.5s
 => => pushing layer c29abcf4219d                                                                                                                                                                                                                                          0.5s
 => => pushing layer 2c9612efc5d6                                                                                                                                                                                                                                          0.5s
 => => pushing layer 3109d88adfae                                                                                                                                                                                                                                          0.5s
 => => pushing layer de48e899da14                                                                                                                                                                                                                                          0.5s
 => => pushing layer ac6f92babc19                                                                                                                                                                                                                                          0.5s
 => => pushing layer 4032548fa860                                                                                                                                                                                                                                          0.5s
 => => pushing layer 03dbc70c199c                                                                                                                                                                                                                                          0.5s
 => => pushing layer f806f4073cf5                                                                                                                                                                                                                                          0.5s
 => => pushing layer 462c78d623f8                                                                                                                                                                                                                                          0.5s
 => => pushing layer c0f63f893bfb                                                                                                                                                                                                                                          0.5s
 => => pushing layer 987d9826e606                                                                                                                                                                                                                                          0.5s
 => => pushing layer d2dc08645627                                                                                                                                                                                                                                          0.5s
 => => pushing layer 16bf97960ee7                                                                                                                                                                                                                                          0.5s
 => => pushing layer 190990e6fae4                                                                                                                                                                                                                                          0.5s
 => => pushing layer e44afd48b6d5                                                                                                                                                                                                                                          0.5s
 => => pushing layer 799e7111d6d4                                                                                                                                                                                                                                          0.5s
 => => pushing layer a72a7e555fe1                                                                                                                                                                                                                                          0.5s
 => => pushing layer b8f8aeff56a8                                                                                                                                                                                                                                          0.5s
 => => pushing layer 687890749166                                                                                                                                                                                                                                          0.5s
 => => pushing layer 2f77733e9824                                                                                                                                                                                                                                          0.5s
 => => pushing layer 97041f29baff                                                                                                                                                                                                                                          0.5s
 => pushing 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit with docker                                                                                                                                                                  0.4s
 => => pushing layer e78ea794e900                                                                                                                                                                                                                                          0.4s
 => => pushing layer 04203156884c                                                                                                                                                                                                                                          0.4s
 => => pushing layer 5f70bf18a086                                                                                                                                                                                                                                          0.4s
 => => pushing layer c29abcf4219d                                                                                                                                                                                                                                          0.4s
 => => pushing layer 2c9612efc5d6                                                                                                                                                                                                                                          0.4s
 => => pushing layer 3109d88adfae                                                                                                                                                                                                                                          0.5s
 => => pushing layer de48e899da14                                                                                                                                                                                                                                          0.5s
 => => pushing layer ac6f92babc19                                                                                                                                                                                                                                          0.5s
 => => pushing layer 4032548fa860                                                                                                                                                                                                                                          0.5s
 => => pushing layer 03dbc70c199c                                                                                                                                                                                                                                          0.5s
 => => pushing layer f806f4073cf5                                                                                                                                                                                                                                          0.5s
 => => pushing layer 462c78d623f8                                                                                                                                                                                                                                          0.5s
 => => pushing layer c0f63f893bfb                                                                                                                                                                                                                                          0.5s
 => => pushing layer 987d9826e606                                                                                                                                                                                                                                          0.5s
 => => pushing layer d2dc08645627                                                                                                                                                                                                                                          0.5s
 => => pushing layer 16bf97960ee7                                                                                                                                                                                                                                          0.5s
 => => pushing layer 190990e6fae4                                                                                                                                                                                                                                          0.5s
 => => pushing layer e44afd48b6d5                                                                                                                                                                                                                                          0.5s
 => => pushing layer 799e7111d6d4                                                                                                                                                                                                                                          0.5s
 => => pushing layer a72a7e555fe1                                                                                                                                                                                                                                          0.5s
 => => pushing layer b8f8aeff56a8                                                                                                                                                                                                                                          0.5s
 => => pushing layer 687890749166                                                                                                                                                                                                                                          0.5s
 => => pushing layer 2f77733e9824                                                                                                                                                                                                                                          0.5s
 => => pushing layer 97041f29baff                                                                                                                                                                                                                                 0.0s

Docker image hash

ubuntu@ip-172-31-88-222:~$ docker inspect 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f | jq -r '.[].Id'
sha256:b762d9b76d272ac22305cbcbefef2a28ecb6d70baffecd008e75913d0620ddc8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided bake.hcl and reproduce the issue using the docker buildx bake command and cache settings shown in the logs. Compare cache imports and exports when both targets are built but only one is pushed. Done means consecutive runs reuse the expected cache instead of reporting a miss.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.