docker / docker/bake-action

The cache generated by the last runner of a multi-platform image will overwrite the caches produced by the previous runners

Open
#329 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status/triage
Dominant language
TypeScript
Stars
301
Forks
43
Avg merge
2d 5h
Merged PRs (30d)
20

Description

Contributing guidelines
I've found a bug, and:
  • The documentation does not mention anything about my problem
  • There are no open or closed issues that are related to my problem
Description

When building a multi-platform image in parallel using multiple GitHub Actions runners, the cache generated by the last runner will overwrite the caches produced by the previous runners, resulting in only a single architecture's cache being available for subsequent builds.

You can observe from my multiple executions of the same workflow run that each time, only one architecture was able to utilize the cache.
https://github.com/vivodi/bake-action-demo/actions/runs/15037459653

Image

Expected behaviour

When using multiple runners to build multi-platform images in parallel, the caches for different architectures can coexist harmoniously without conflicts.

Actual behaviour

Only the cache corresponding to the architecture that completes its build last will be retained.

Repository URL

https://github.com/vivodi/bake-action-demo

Workflow run URL

https://github.com/vivodi/bake-action-demo/actions/runs/15037459653

YAML workflow
name: Docker Image CI

on:
  push:
    branches:
      - develop
    tags:
      - v*.*.*
  pull_request:
  merge_group:
permissions: { }
concurrency:
  group: ${{ github.workflow }}-${{ github.event.number || github.ref || github.run_id }}
  cancel-in-progress: true
env:
  REGISTRY_IMAGE: ${{ vars.DOCKER_IMAGE_NAMESPACE }}/flexget
  GHCR_IMAGE: ghcr.io/${{ github.repository_owner }}/flexget
jobs:
  build:
    name: Docker build
    runs-on: ${{ matrix.operating-system }}
    strategy:
      fail-fast: false
      matrix:
        include:
          - platform: linux/amd64
            operating-system: ubuntu-latest
          - platform: linux/arm64
            operating-system: ubuntu-24.04-arm
    steps:
      - name: Prepare
        run: |
          platform=${{ matrix.platform }}
          echo PLATFORM_PAIR=${platform//\//-} >> $GITHUB_ENV

      - name: Docker meta
        id: meta
        uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 # v5
        with:
          images: |
            ${{ env.REGISTRY_IMAGE }}
            ${{ env.GHCR_IMAGE }}
          tags: |
            type=ref,event=branch
            type=pep440,pattern={{version}}
            type=pep440,pattern={{major}}.{{minor}}
            type=pep440,pattern={{major}}

      - name: Rename meta bake definition file
        run: |
          mv ${{ steps.meta.outputs.bake-file }} ${{ runner.temp }}/bake-meta.json

      - name: Upload meta bake definition
        if: matrix.platform == 'linux/amd64'
        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
        with:
          name: bake-meta
          path: ${{ runner.temp }}/bake-meta.json
          if-no-files-found: error
          retention-days: 1

      - name: Login to Docker Hub
        if: github.event_name == 'push'
        uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3
        with:
          username: ${{ vars.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_PASSWORD }}

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3 # zizmor: ignore[cache-poisoning]

      - name: Build
        id: bake
        uses: docker/bake-action@76f9fa3a758507623da19f6092dc4089a7e61592 # v6
        with:
          files: |
            ./docker-bake.hcl
            cwd://${{ runner.temp }}/bake-meta.json
          targets: all
          set: |
            *.cache-from=type=gha
            *.cache-to=type=gha,mode=max
            *.tags=
            *.platform=${{ matrix.platform }}
            *.output=type=image,name=${{ env.REGISTRY_IMAGE }},push-by-digest=true,push=${{github.event_name == 'push'}}

      - name: Export digest
        run: |
          mkdir -p ${{ runner.temp }}/digests
          digest="${{ fromJSON(steps.bake.outputs.metadata).all['containerimage.digest'] }}"
          touch "${{ runner.temp }}/digests/${digest#sha256:}"

      - name: Upload digest
        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
        with:
          name: digests-${{ env.PLATFORM_PAIR }}
          path: ${{ runner.temp }}/digests/*
          if-no-files-found: error
          retention-days: 1

  merge:
    if: github.event_name == 'push'
    needs:
      - build
    runs-on: ubuntu-latest
    permissions:
      packages: write # to push to ghcr.io
    steps:
      - name: Download meta bake definition
        uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4
        with:
          name: bake-meta
          path: ${{ runner.temp }}

      - name: Download digests
        uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4
        with:
          path: ${{ runner.temp }}/digests
          pattern: digests-*
          merge-multiple: true

      - name: Log in to Docker Hub
        uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3
        with:
          username: ${{ vars.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_PASSWORD }}

      - name: Login to GitHub Container Registry
        uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ github.token }}

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3 # zizmor: ignore[cache-poisoning]

      - name: Create manifest list and push
        working-directory: ${{ runner.temp }}/digests
        run: |
          docker buildx imagetools create $(jq -cr '.target."docker-metadata-action".tags |
            map(select(startswith("${{ env.REGISTRY_IMAGE }}") or startswith("${{ env.GHCR_IMAGE }}")) | "-t " + .) |
            join(" ")' ${{ runner.temp }}/bake-meta.json) $(printf '${{ env.REGISTRY_IMAGE }}@sha256:%s ' *)
Workflow logs
#21 exporting to GitHub Actions Cache
#21 preparing build cache for export
#21 writing layer sha256:090e07cdd718dbda93d9f844d2d238f14d3b78a9beb20802dd68b334ae933ca8
#21 writing layer sha256:090e07cdd718dbda93d9f844d2d238f14d3b78a9beb20802dd68b334ae933ca8 0.2s done
#21 writing layer sha256:144b496cef01f8fb2d967649e86baaf913809984a526a8404dc9de1de3b887d3
#21 writing layer sha256:144b496cef01f8fb2d967649e86baaf913809984a526a8404dc9de1de3b887d3 3.5s done
#21 writing layer sha256:3b40a9883da8a5ff7af537ff613f97f169a24c78f86b8a461456d7318c665da4
#21 writing layer sha256:3b40a9883da8a5ff7af537ff613f97f169a24c78f86b8a461456d7318c665da4 8.4s done
#21 writing layer sha256:7a072645e9a35bbd11a86d494a31b8043621ec1afb03a5d2aba8627436630f06
#21 writing layer sha256:7a072[645](https://github.com/vivodi/bake-action-demo/actions/runs/15037459653/job/42262138983#step:8:653)e9a35bbd11a86d494a31b8043621ec1afb03a5d2aba8627436630f06 0.3s done
#21 writing layer sha256:80d424cdff5e3f72d996702ae5a5538744a5007a7238787dc45588da26086f89
#21 writing layer sha256:80d424cdff5e3f72d996702ae5a5538744a5007a7238787dc45588da26086f89 0.3s done
#21 writing layer sha256:89f69128dc8008f8a290d44b0602ac8b3684a972d9a276fd2dfa3babf39c4706
#21 writing layer sha256:89f69128dc8008f8a290d44b0602ac8b3684a972d9a276fd2dfa3babf39c4706 2.6s done
#21 writing layer sha256:8c3ab6dc13c1beb8ab550a572300043b080a1f83c1112d5f75c4a359fa1738a2
#21 writing layer sha256:8c3ab6dc13c1beb8ab550a572300043b080a1f83c1112d5f75c4a359fa1738a2 1.4s done
#21 writing layer sha256:a6efc32074759efa372ff772f6ae64f9ab92888413cad13[646](https://github.com/vivodi/bake-action-demo/actions/runs/15037459653/job/42262138983#step:8:654)07ff5771af665d
#21 writing layer sha256:a6efc32074759efa372ff772f6ae64f9ab92888413cad1364607ff5771af665d 7.9s done
#21 writing layer sha256:c99ff282ead22e0178c9b70e17121a386ecb92874b716a9ae43ce46700285226
#21 writing layer sha256:c99ff282ead22e0178c9b70e17121a386ecb92874b716a9ae43ce46700285226 1.7s done
#21 writing layer sha256:ceca85c73e46cd320504cc3617c8d9c0800c291922f6d593a107247a46b6070b
#21 writing layer sha256:ceca85c73e46cd320504cc3617c8d9c0800c291922f6d593a107247a46b6070b 2.5s done
#21 preparing build cache for export 38.4s done
#21 DONE 38.4s
BuildKit logs

Additional info

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the supplied workflow YAML, focusing on the parallel matrix jobs and their *.cache-from and *.cache-to settings; compare the resulting cache behavior across linux/amd64 and linux/arm64. Use the linked workflow run and its BuildKit export logs as the baseline. Done means caches from both architectures remain available for subsequent builds instead of the last runner replacing the others.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, github-actions
Domain
build-system, ci-cd, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.