Frequent connection errors and hung github action

Hey, I wrote a pipeline today to build and publish two images to GHCR.

At the final step, It is frequently getting stuck when pushing the image to GHCR with errors like,

#19 14.15 error: failed to copy: failed to do request: Put "https://ghcr.io/v2/instadapp/interop-validator/blobs/upload/64e2da4e-adb4-46f3-be0a-1a8a80688011?digest=sha256%3Ab2288d6f501f153bbc4906079f3278b2658c91b557c14a2ccaf774eb3f8cf220": use of closed network connection

and

#19 13.57 error: failed to copy: failed to do request: Put "https://ghcr.io/v2/instadapp/interop-sequencer/blobs/upload/abef1bd5-b3cd-4914-b7bc-c5171e12ba9d?digest=sha256%3A1b623b54e74dada99816481bef3d7195a8b8d6d5ab27242bfbefb13170d9a82a": write tcp 172.17.0.2:54880->140.82.113.33:443: write: connection reset by peer

These actions have been going on for hours at this point. I believe github.com/docker/docker-push-action is supposed to retry but that doesn't seem to be working. I think the issue is with buildkit but I'll create a issue on the downstream repo if that's not the case.

Configuration

name: Build Docker Images

on:
  push:
    branches:
      # DO NOT USE `/` in branches that'll be built and pushed to ghcr
      - 'master'

env:
  REGISTRY: ghcr.io
  SEQUENCER_IMAGE_NAME: ${{ github.repository }}-sequencer
  VALIDATOR_IMAGE_NAME: ${{ github.repository }}-validator

jobs:
  build_sequencer_image:
    name: Build Sequencer Docker Image
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1

      - name: Login to Container Registry
        uses: docker/login-action@v1.10.0
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract Metadata for Docker
        id: meta
        uses: docker/metadata-action@v3
        with:
          images: ${{ env.REGISTRY }}/${{ env.SEQUENCER_IMAGE_NAME }}
          flavor: |
            latest=true
          tags: |
            event=push,type=sha,format=short
            event=tag,type=ref

      - name: Build Docker Image
        uses: docker/build-push-action@v2
        with:
          push: true
          file: Dockerfile.sequencer
          platforms: linux/amd64
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha, scope=interop-sequencer
          cache-to: type=gha, scope=interop-sequencer, mode=max
    outputs:
      full_image_path: ${{ steps.meta.outputs.tags }}
  build_validator_image:
    name: Build Validator Docker Image
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1

      - name: Login to Container Registry
        uses: docker/login-action@v1.10.0
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract Metadata for Docker
        id: meta
        uses: docker/metadata-action@v3
        with:
          images: ${{ env.REGISTRY }}/${{ env.VALIDATOR_IMAGE_NAME }}
          flavor: |
            latest=true
          tags: |
            event=push,type=sha,format=short
            event=tag,type=ref

      - name: Build Docker Image
        uses: docker/build-push-action@v2
        with:
          push: true
          file: Dockerfile.validator
          platforms: linux/amd64
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha, scope=interop-validator
          cache-to: type=gha, scope=interop-validator, mode=max
    outputs:
      full_image_path: ${{ steps.meta.outputs.tags }}

Start with the workflow configuration and the Dockerfile.sequencer and Dockerfile.validator entries that use docker/build-push-action@v2 to push images to GHCR. Reproduce the intermittent upload errors and compare the action's retry behavior with the reported BuildKit failures; done means identifying a reproducible cause and a confirmed fix or clear handoff.