[BUG] changes to docker system prune with buildkit, now deletes downloaded base images

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

(originally reported for docker compose https://github.com/docker/compose/issues/11346, they advised raise here)

In the days pre buildkit, all base images were tagged and stored in the main docker images list.

After buildkit, this no longer happens. Base images are downloaded into the buildkit cache.

https://github.com/moby/buildkit/issues/1537
https://github.com/docker/compose/issues/10206

This is problematic if you have large images and a slow download link and you run out of diskspace. Say you have a 10GB image and a 1MB/s download speed.

Previously, if you did docker system prune, the base image would be preserved, and you have easy control over whether to delete it or not at a later time

Now, if you do docker system prune, it will delete the 10GB image and you will start swearing as you wait 3 hours to redownload it.

If you have a look into the buildkit cache, there doesn't seem to be a lot of controls over which images to keep and which to prune. I don't see a way in docker buildx du to "keep base images".

Ideally there would be an option to tell compose/docker build to "export" base images into the main docker images list to be compatible with the previous behaviour.

If that is not possible, how can we clean the build cache while keeping downloaded base images like we did before?

Expected behaviour

see description

Actual behaviour

see description

Buildx version

github.com/docker/buildx v0.11.2 9872040

Docker info

No response

Builders list

NAME/NODE DRIVER/ENDPOINT STATUS  BUILDKIT             PLATFORMS
default * docker                                       
  default default         running v0.11.7+d3e6c1360f6e linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386

Configuration

n/a

Build logs

No response

Additional info

No response

Start by reproducing the reported docker system prune behavior with a BuildKit builder and compare it with the cache information shown by docker buildx du. Determine whether the intended fix is an export option or a cache-retention control; done means downloaded base images can be retained while other build cache is pruned.