Github actions cache (experimental) still seems to cache python packages after docker layer change

We have a workflow which includes the following step to build and push our image - docker/build-push-action@v3:

      - name: Build and push
        uses: docker/build-push-action@v3
        with:
          push: true
          tags: ${{ inputs.image_name && format(',{0}:{1}', inputs.image_name, steps.tags.outputs.tag1) || '' }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
          provenance: false
          build-args: |
            ${{ secrets.build-args }}
            ${{ inputs.build-args }}
          secrets: |
            ${{secrets.build-secrets}}
          ssh: |
            default=${{ env.SSH_AUTH_SOCK }}

In the Dockerfile we install a few private python packages, here's a shortened version of our Dockerfile showing relevant bits:

FROM python:3.10


RUN pip install -U pip \
    && pip install 'poetry>=1.0.0' \
    && poetry config virtualenvs.create false \
    && poetry install $(test "$BUILD_ENV" == ci && echo "--no-dev") --no-interaction --no-ansi

ARG GIT_HASH

# Install python packages
RUN --mount=type=ssh \
    mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts \
    && poetry run pip install "git+ssh://git@github.com/**REDACTED**/python-files.git#egg=**REDACTED**&subdirectory=**REDACTED**" \
    && poetry run pip install "git+ssh://git@github.com/**REDACTED**/python-files.git#egg=**REDACTED**&subdirectory=**REDACTED**"


CMD python main.py

We use ARG GIT_HASH which will change on each github branch build, this has been added to trigger a cache miss when installing the python packages immediately after as per the official docs - If a layer changes, all other layers that come after it are also affected. When the layer with the ARG GIT_HASH command gets invalidated, all layers that follow will need to run again, too, to ensure a fresh up to date copy of the packages are installed rather than use the cache.

However this doesn't appear to be working and the image is still being built using older versions of the packages.

It only works if I completely comment out the cache-from and cache-to lines to stop using the github actions cache.

Snippet from Docker build log on github actions:

#12 [5/7] RUN --mount=type=ssh     mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts && poetry run pip install "git+ssh://git@github.com/**REDACTED**/python-files.git#egg=**REDACTED**&subdirectory=**REDACTED**" && poetry run pip install "git+ssh://git@github.com/**REDACTED**/python-files.git#egg=**REDACTED**&subdirectory=**REDACTED**" && poetry run pip install "git+ssh://git@github.com/**REDACTED**/
#12 extracting sha256:7d300de960bdd6899dd0ea1687b4d4ab68198341d0e60957463099eafd9e1eab done
#12 extracting sha256:0fc41ee805e3e6029f57a8202d1dcb6a5b15701070236117318a4ff67c0735f2
#12 extracting sha256:0fc41ee805e3e6029f57a8202d1dcb6a5b15701070236117318a4ff67c0735f2 0.2s done
#12 DONE 19.7s

#13 [6/7] COPY app /app
#13 DONE 1.8s

#14 [7/7] WORKDIR /app
#14 DONE 0.0s

#15 exporting to image
#15 exporting layers 0.1s done

Any ideas why this could be?

Start with the shown GitHub Actions docker/build-push-action step, its cache-from/cache-to settings, and the Dockerfile layer containing ARG GIT_HASH and the SSH-mounted pip installs. Compare the build log and cache behavior with and without the GitHub Actions cache. Done means the cause of the stale Python packages is identified and the cache behavior is corrected or clearly documented.