[v0.13] create --append with missing builder changed from warn to error

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Hi,
We are extensively using buildx with remote buildkit nodes, notably for its speed and scheduling of build on native CPU architectures (arm64 build on arm64 VMs, same for amd64)

Before the version 0.13.0, we used the following syntax in our CI/CD pipelines

docker buildx create \
  --append \
  --name buildkit \
  --driver remote \
  --driver-opt cacert=/certs/ca.pem,cert=/certs/cert.pem,key=/certs/key.pem \
  --platform arm64 \
  tcp://buildkit-arm64.buildkit.svc.cluster.local:1234 ;
  
  docker buildx create \ 
  --append \
  --name buildkit \
  --driver remote \
  --driver-opt cacert=/certs/ca.pem,cert=/certs/cert.pem,key=/certs/key.pem \
  --platform amd64 \
  tcp://buildkit-amd64.buildkit.svc.cluster.local:1234 ;

 docker buildx build \ 
  --platform amd64,arm64 \
...

Note that we use the same syntax docker buildx create --append both time. This is not well documentend but worked nicely, with a warning for the first command

WARNING: failed to find "buildkit" for append, creating a new instance instead buildkit

With the 0.13.0 release, this behavious was broken, with the following error

ERROR: failed to find instance "buildkit" for append

It seems that this is linked to a refactoring of the create command on this commit (@crazy-max )

Since buildx is automatically shipped with docker, the 0.13.0 release was unknowingly shipped with a daily rebuild of docker:24-cli and broke our CI/CD.

While this is an undocumented feature, and we might want to explicitely fail on this usage, I find the usage of an idempotent command to add node to a builder very useful.

Indeed, in our case, we dynamically generate the CI depending on how many CPU archs we want to build, either 1 or more. So we would be forced to implement a logic like "if this is the first command, do not put --append, otherwise do. While this is doable, idempotent command are truly useful. Imagine a world without kubectl apply but only kubectl create and kubectl patch !

If you agree with my proposal, I might draft a PR, seems it looks not that complicated to reintroduce this logic. What do you think ?

Expected behaviour

docker buildx create \
  --append \
  --name buildkit \
  --driver remote \
  --driver-opt cacert=/certs/ca.pem,cert=/certs/cert.pem,key=/certs/key.pem \
  --platform arm64 \
  tcp://buildkit-arm64.buildkit.svc.cluster.local:1234 ;
  WARNING: failed to find "buildkit" for append, creating a new instance instead buildkit
  
  docker buildx create \ 
  --append \
  --name buildkit \
  --driver remote \
  --driver-opt cacert=/certs/ca.pem,cert=/certs/cert.pem,key=/certs/key.pem \
  --platform amd64 \
  tcp://buildkit-amd64.buildkit.svc.cluster.local:1234 ;

Actual behaviour

docker buildx create \
  --append \
  --name buildkit \
  --driver remote \
  --driver-opt cacert=/certs/ca.pem,cert=/certs/cert.pem,key=/certs/key.pem \
  --platform arm64 \
  tcp://buildkit-arm64.buildkit.svc.cluster.local:1234 ;
ERROR: failed to find instance "buildkit" for append

Buildx version

v0.13.0

Docker info

No response

Builders list

N/A

Configuration

FROM alpine

Build logs

No response

Additional info

No response

Start by reproducing the v0.13.0 behavior with the documented docker buildx create --append --name buildkit sequence and inspect the refactoring in commit ceb5bc807ce92d48074713434d9cbb56a3500d01. Done means deciding and testing whether a missing builder with --append should create a new instance with a warning instead of returning an error.