docker compose up fails to pull image from private repository unless DOCKER_BUILDKIT=0 is set

I know this has been reported in the past and ostensibly fixed, but on my Mac with the following docker versions I run into an issue consistently where docker cannot pull from a private repository when running docker compose. Setting DOCKER_BUILDKIT=0 in the environment before running fixes this problem.

Docker info:

$ docker version
Client:
 Cloud integration: 1.0.14
 Version:           20.10.6
 API version:       1.41
 Go version:        go1.16.3
 Git commit:        370c289
 Built:             Fri Apr  9 22:46:57 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.6
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8728dd2
  Built:            Fri Apr  9 22:44:56 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ docker-compose version
docker-compose version 1.29.1, build c34c88b2
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

A sample run looks like:

Building projectA
[+] Building 0.8s (6/9)
 => [internal] load build definition from Dockerfile                                                                                                                                                                                           0.0s
 => => transferring dockerfile: 37B                                                                                                                                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                                                                                                                0.0s
 => [internal] load metadata for private.repo/local-docker/base/ubuntu-java-base:8u121                                                                                                                                         0.7s
 => [auth] local-docker/base/ubuntu-java-base:pull token for private.repo                                                                                                                                                      0.0s
 => [internal] load build context                                                                                                                                                                                                              0.0s
 => ERROR [1/4] FROM private.repo/local-docker/base/ubuntu-java-base:8u121@sha256:cc14d8f722dd428ff4bb81f91ace16d3cc201fc2e81cf673b1178d18f885e3fc                                                                             0.0s
 => => resolve private.repo/local-docker/base/ubuntu-java-base:8u121@sha256:cc14d8f722dd428ff4bb81f91ace16d3cc201fc2e81cf673b1178d18f885e3fc                                                                                   0.0s
------
 > [1/4] FROM private.repo/local-docker/base/ubuntu-java-base:8u121@sha256:cc14d8f722dd428ff4bb81f91ace16d3cc201fc2e81cf673b1178d18f885e3fc:
------
failed to load cache key: invalid empty config file resolved for private.repo/local-docker/base/ubuntu-java-base:8u121@sha256:cc14d8f722dd428ff4bb81f91ace16d3cc201fc2e81cf673b1178d18f885e3fc

After disabling buildkit:

Building projectA
Sending build context to Docker daemon  44.03kB
Step 1/5 : from private.repo/local-docker/base/ubuntu-java-base:8u121
8u121: Pulling from local-docker/base/ubuntu-java-base
ae79f2514705: Pull complete
c59d01a7e4ca: Pull complete
41ba73a9054d: Pull complete
f1bbfd495cc1: Pull complete
0c346f7223e2: Pull complete
6d49f1b8c7dc: Pull complete
49c116ee4203: Pull complete
db2b55776c91: Pull complete
c939b587f4b0: Pull complete
57f71c3cdf28: Pull complete
d46ba4a611f9: Pull complete
97eb6206c1f9: Pull complete
Digest: sha256:cc14d8f722dd428ff4bb81f91ace16d3cc201fc2e81cf673b1178d18f885e3fc
Status: Downloaded newer image for private.repo/local-docker/base/ubuntu-java-base:8u121

Not sure if this is a regression or a new bug.

Start by reproducing the failure with the Docker and docker-compose versions and private image shown, then compare the BuildKit path with the legacy path when DOCKER_BUILDKIT=0. Done means docker compose can pull the private image through BuildKit without the environment workaround; the issue does not name files or tests to begin with.