Add upgrade command

Add buildx upgrade command that can be used to update BuildKit version or properties of an existing builder instance. This would work for container and kubernetes drivers.

This command will shut down the buildkit instance (docker buildx stop) , ideally waiting before running builds have completed unless forced, update properties and then boot a new builder. The internal state volume of the builder will remain the same. Currently, a cumbersome way to achieve similar behavior is to use docker buildx rm --keep-state followed by a new docker buildx create.

Capabilities

Without parameters upgrade command will try to upgrade the buildkit image. By default that would meanmoby/buildkit:buildx-stable-1. The command should report actual buildkit version though, and ideally skip to process if no newer image is available.

The following flags are possible from buildx create:

--driver-opt k=v will change the driver-opt values for instance (builder state). This can be used to switch to a different BuildKit image, expose new environmental variables(eg. for proxy or for progress clipping limits), modify default-load etc. All driver opt can be changed. If the user wants to remove the existing driver-opt, they can be set with an empty value.

--buildkitd-flags

--buildkitd-config new buildkit TOML config.

New flags:

--update-config takes a new TOML config that is then merged with the existing config. This should make it easy to change allowed entitlements, configure GC, update mirror config, modify CDI device rules etc. with existing drivers. Eg. if a project depends on builder with specific entitlements or CDI devices, it should be easy for them to put in their readme that one needs to run docker buildx upgrade <something> and then they have a valid builder.

Note that the endpoint of the builder can't be changed. I think it would be too messy otherwise.

Follow-ups

If the upgrade fails(builder fails to boot), we could try to roll back to the previous settings/image.

For the default buildkit image moby/buildkit:buildx-stable-1 we could periodically keep track when it gets updated. If we notice that the builder is not using the latest version and can be upgraded to a new buildkit we could show a warning in ls and inpsect.

We could consider special flags for some of the buildkit options, like --allow-entitlement instead of --buildkit-flags "--allow-insecure-entitlement=network.host", but I think these would need to be added to create as well , so not directly related to this. Changing GC settings could probably also be simpler than it is today.

Is upgrade the right name for this? update ? Something else more tweaked to changing the config?

@crazy-max @colinhemmings

Start by comparing the existing buildx create, buildx stop, and buildx rm --keep-state flows for container and Kubernetes drivers. Define the command name and supported flags, then determine how to preserve state while updating the image and configuration. Done means an existing builder can be upgraded without changing its endpoint, with the requested settings applied and the builder restarted.