dockerutil.BuildEmptyImage() fails with bad API version

I've got a CI/CD pipeline that's run occasionally when we rebuild a image and create an optimized version with Slim. Without anything having changed that I'm aware of (same Slim version, same pipeline runner), this process has started failing with:

dockerutil.BuildEmptyImage: dockerapi.BuildImage() error = API error (400): client version 1.12 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version

I thought this might be related to #637, but it seems to be a separate issue, and the same thing happens whether or not I try setting --crt-api-version. Having updated everything to the latest versions, the problem continues. The issue seems to be specific to this environment, as a local build works.

Steps to Reproduce the Problem

1. slim build --http-probe-off --continue-after=enter alpine

(trivial reproduction; of course not our actual image)

cmd=build state=started
cmd=build info=params target.image='alpine' continue.mode='enter' rt.as.user='true' keep.perms='true' tags='' image-build-engine='internal' target.type='image' 
cmd=build state=image.inspection.start
cmd=build info=image id='sha256:05455a08881ea9cf0e752bc48e61bbd71a34c029bb13df01e40e3e70e0d007bd' size.bytes='0' size.human='0 B' 
cmd=build info=image.stack index='0' name='alpine:latest' id='sha256:05455a08881ea9cf0e752bc48e61bbd71a34c029bb13df01e40e3e70e0d007bd' 
cmd=build state=image.inspection.done
cmd=build state=container.inspection.start
time="2024-02-26T18:12:39Z" level=error msg="dockerutil.BuildEmptyImage: dockerapi.BuildImage() error = API error (400): client version 1.12 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version\n / output: "
time="2024-02-26T18:12:39Z" level=fatal msg="slim: failure" error="API error (400): client version 1.12 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version\n" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x5e\ngithub.com/slimtoolkit/slim/pkg/util/errutil.FailOn({0x231e3e0, 0xc0007e62a0})\n\tgithub.com/slimtoolkit/slim/pkg/util/errutil/errutil.go:32 +0x4b\ngithub.com/slimtoolkit/slim/pkg/app/master/inspectors/container.(*Inspector).RunContainer(0xc000464b00)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/inspectors/container/container_inspector.go:430 +0x77a\ngithub.com/slimtoolkit/slim/pkg/app/master/command/build.OnCommand(_, _, {_, _}, _, {_, _}, {_, _}, {0x0, ...}, ...)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/build/handler.go:1132 +0x4632\[ngithub.com/slimtoolkit/slim/pkg/app/master/command/build.glob..func1(0xc0003d35c0)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/build/cli.go:760](http://ngithub.com/slimtoolkit/slim/pkg/app/master/command/build.glob..func1(0xc0003d35c0)/n/tgithub.com/slimtoolkit/slim/pkg/app/master/command/build/cli.go:760) +0x5d51\[ngithub.com/urfave/cli/v2.(*Command](http://ngithub.com/urfave/cli/v2.(*Command)).Run(0x3396180, 0xc0000af480)\n\[tgithub.com/urfave/cli/v2@v2.3.0/command.go:163](http://tgithub.com/urfave/cli/v2@v2.3.0/command.go:163) +0x583\[ngithub.com/urfave/cli/v2.(*App](http://ngithub.com/urfave/cli/v2.(*App)).RunContext(0xc000266d00, {0x233bb88?, 0x340d500}, {0xc00012e000, 0x7, 0x7})\n\[tgithub.com/urfave/cli/v2@v2.3.0/app.go:313](http://tgithub.com/urfave/cli/v2@v2.3.0/app.go:313) +0xaa5\[ngithub.com/urfave/cli/v2.(*App](http://ngithub.com/urfave/cli/v2.(*App)).Run(...)\n\[tgithub.com/urfave/cli/v2@v2.3.0/app.go:224\ngithub.com/slimtoolkit/slim/pkg/app/master.Run()\n\tgithub.com/slimtoolkit/slim/pkg/app/master/app.go:15](http://tgithub.com/urfave/cli/v2@v2.3.0/app.go:224/ngithub.com/slimtoolkit/slim/pkg/app/master.Run()/n/tgithub.com/slimtoolkit/slim/pkg/app/master/app.go:15) +0x45\nmain.main()\n\[tgithub.com/slimtoolkit/slim/cmd/slim/main.go:15](http://tgithub.com/slimtoolkit/slim/cmd/slim/main.go:15) +0x187\n" version="linux/amd64|Transformer|1.40.11|1b271555882eacdfb4e6598d6d0552e9b9b1449b|2024-02-02_01:36:22PM"

Specifications

• Version: 1.40.11
• Platform: linux/amd64 (self-hosted Bitbucket pipeline runner, which is configured for privileged Docker access so Slim can run)
• Output of slim version:

cmd=version info=app container='true' dsimage='false' location='/opt/atlassian/pipelines/agent/build/dist_linux' current='1.40.11' verdict='you have the latest version' cmd='version' version='linux/amd64|Transformer|1.40.11|1b271555882eacdfb4e6598d6d0552e9b9b1449b|2024-02-02_01:36:22PM' 
cmd=version info=host cmd='version' osname='Ubuntu 22.04.3 LTS' osbuild='' version='#18~22.04.1-Ubuntu SMP Wed Jan 10 22:54:16 UTC 2024' release='6.2.0-1018-aws' sysname='Linux' 
cmd=version info=docker server.version='25.0.3' architecture='x86_64' cmd='version' name='a640edfcc04d' kernel.version='6.2.0-1018-aws' operating.system='Alpine Linux v3.19 (containerized)' ostype='linux' 
cmd=version info=dclient build.time='2024-02-06T21:13:08.000000000+00:00' git.commit='f417435' cmd='version' api.version='1.44' min.api.version='1.24' 
app='slim' message='GitHub Discussions' info='https://github.com/slimtoolkit/slim/discussions'
app='slim' message='Join the CNCF Slack channel to ask questions or to share your feedback' info='https://cloud-native.slack.com/archives/C059QP1RH1S'
app='slim' message='Join the Discord server to ask questions or to share your feedback' info='https://discord.gg/9tDyxYS'
app='slim' message='Join the Gitter channel to ask questions or to share your feedback' info='https://gitter.im/docker-slim/community'

Start at dockerutil.BuildEmptyImage and dockerapi.BuildImage, then trace the reported caller in pkg/app/master/inspectors/container/container_inspector.go around line 430. Reproduce slim build --http-probe-off --continue-after=enter alpine against the reported Docker 25.0.3 environment and compare the client API version handling. Done means the build no longer sends client version 1.12 to the Docker server.