Exported build cache randomly missing some platforms

Start with the linked .github/workflows/docker-release.yml and Dockerfile, then rerun the two referenced GitHub Actions builds using the documented registry cache settings. Compare cache export and import results for linux/amd64, linux/arm/v7, and linux/arm64, including the alternative gha cache case. Done means the cause of the missing platform caches is identified and repeated runs reliably restore all three platforms.

TL;DR

https://github.com/moby/buildkit/issues/2822#issuecomment-1113920626

In https://github.com/moby/buildkit/issues/2758#issuecomment-1088288408, @tonistiigi said that If someone has something similar with a single node then create a separate issue with a reproducer so here it is.

I am facing almost the same issue but what I use is GitHub Actions which should only have a single-node builder.

In my case, I need to build for 3 platforms (linux/amd64, linux/arm/v7, linux/arm64). Each time, there are randomly 1 or 2 platforms entirely unable to fetch their cache and must start over.

Build settings

      - name: Build and push Docker image (ordinary version)
        uses: docker/build-push-action@v2
        with:
          context: .
          push: true
          tags: ${{ steps.meta-ordinary.outputs.tags }}
          labels: ${{ steps.meta-ordinary.outputs.labels }}
          platforms: linux/amd64,linux/arm/v7,linux/arm64
          cache-from: type=registry,ref=${{ secrets.DOCKER_USERNAME }}/rsshub:buildcache
          cache-to: type=registry,ref=${{ secrets.DOCKER_USERNAME }}/rsshub:buildcache,mode=max

/usr/bin/docker buildx build 
--cache-from type=registry,ref=***/rsshub:buildcache 
--cache-to type=registry,ref=***/rsshub:buildcache,mode=max 
--iidfile /tmp/docker-build-push-AxluK5/iidfile 
--label org.opencontainers.image.title=RSSHub 
--label org.opencontainers.image.description=🍰 Everything is RSSible 
--label org.opencontainers.image.url=https://github.com/Rongronggg9/RSSHub 
--label org.opencontainers.image.source=https://github.com/Rongronggg9/RSSHub 
--label org.opencontainers.image.version=latest 
--label org.opencontainers.image.created=2022-04-24T03:45:12.333Z 
--label org.opencontainers.image.revision=16de07893dacaaf7382fc985ccd43f3f5e587646 
--label org.opencontainers.image.licenses=MIT 
--platform linux/amd64,linux/arm/v7,linux/arm64 
--tag ***/rsshub:latest 
--tag ***/rsshub:2022-04-24 
--metadata-file /tmp/docker-build-push-AxluK5/metadata-file 
--push .

(I added linebreaks to make it readable.)

The node

  Name:   builder-798d7bea-6cd7-454f-baa6-32276a2b4c26
  Driver: docker-container
  
  Nodes:
  Name:      builder-798d7bea-6cd7-454f-baa6-32276a2b4c260
  Endpoint:  unix:///var/run/docker.sock
  Status:    running
  Flags:     --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host
  Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

  {
    "name": "builder-798d7bea-6cd7-454f-baa6-32276a2b4c26",
    "driver": "docker-container",
    "node_name": "builder-798d7bea-6cd7-454f-baa6-32276a2b4c260",
    "node_endpoint": "unix:///var/run/docker.sock",
    "node_status": "running",
    "node_flags": "--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host",
    "node_platforms": "linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/amd64/v4,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6"
  }

How to reproduce

1. Delete all relevant build caches on Docker Hub to avoid potential interference. (optional)
2. Trigger action run attempt #1, which should have the build caches of all 3 platforms pushed to the registry.
3. Wait until finished, then re-run the job, trigger action run attempt #2, which should have all 3 platforms get cache hits, but only linux/amd64 got.

FYI

Workflow YAML
Dockerfile

This issue seems to exist when the cache type is gha, but I am not so sure if this is because of the cache size limit of GitHub Actions.