GitHub Actions Cache doesn't work unless the image is pushed into the registry

Reproduce the workflow with Dockerfile, docker/build-push-action@v3, load:true, and the gha cache settings; compare repeated runs with push:true using the supplied build logs. Trace BuildKit's GitHub Actions cache import/export path and determine whether loaded images should reuse the exported cache; done means the behavior is fixed or clearly documented with a regression test.

In my project I have configured Docker containers, which bundle all the application dependencies and are used locally by the developers to keep their local machines reasonably clean. I wanted to integrate these images into CI/CD to ensure maximum repeatability and avoid repeating myself when defining certain commands.

I have a custom action, which builds the "builder" image, defined as follows:

    - name: Build builder
      uses: docker/build-push-action@v3
      with:
        file: Dockerfile
        context: '.'
        tags: my-builder-image
        load: true
        cache-from: type=gha,scope=${{ github.workflow }}
        cache-to: type=gha,mode=max,scope=${{ github.workflow }}

It creates the image in the scope of Buildx builder and then loads it into Github Action runner's Docker instance.

Then in my jobs I am using the built image to execute the command in the

    - uses: addnab/docker-run-action@v3
      with:
        image: my-builder-image
        options: -w /project -v ${{ github.workspace }}:/project
        run: pylint

The build-push-action seems to call the correct docker command, reads the GHA cache and pushes it after building all the layers:

/usr/bin/docker buildx build --cache-from type=gha,scope=Verify --cache-to type=gha,mode=max,scope=Verify ...

#7 importing cache manifest from gha:12854520869560610746
#7 DONE 0.1s

#15 exporting cache
#15 preparing build cache for export 0.0s done
#15 writing layer sha256:1d578c655d03d719eb075b5759fb1d9ee02065492a5d54db2fa1c390898c3366
#15 writing layer sha256:1d578c655d03d719eb075b5759fb1d9ee02065492a5d54db2fa1c390898c3366 1.8s done
#15 writing layer sha256:927062cd74cc90148418b13fa6614da6c2f9f6c8add3b38fb39ed933d513f76e
#15 writing layer sha256:927062cd74cc90148418b13fa6614da6c2f9f6c8add3b38fb39ed933d513f76e 0.1s done
#15 writing layer sha256:9e01ad9582c1ac5f39f74b04a0a415939a41f96c342cad821a034b1482aaf2ce 0.1s done
#15 writing layer sha256:a9a04eb6545e2a381d6c68e70000bebb363068182391046868fbb320fb8cba00
#15 writing layer sha256:a9a04eb6545e2a381d6c68e70000bebb363068182391046868fbb320fb8cba00 0.2s done
#15 writing layer sha256:f12926aa5844296bacf32e0eac8c15146f9d1dcbaeefbef4de8c9996c0577c47
#15 writing layer sha256:f12926aa5844296bacf32e0eac8c15146f9d1dcbaeefbef4de8c9996c0577c47 0.1s done
#15 DONE 2.8s

However, subsequent executions of the job on the same branch do not use this cache and rebuild all the layers

When I replace load: true with push: true and put the builder image into registry, the caching works as expected. However, it also pollutes my organization's package registry with unnecessary images.

I didn't find any mention on why the caching shouldn't work when pushing is not enabled - is it by design or is this a bug?