aws / aws/aws-sam-cli

Bug: --use-buildkit doesn't bypass Engine-API pre-check — unusable without a local Docker daemon

Open
#8,946 5 comments 0 reactions 0 assignees View on GitHub
area/build
Dominant language
Python
Stars
6.7k
Forks
1.2k
Avg merge
1d 10h
Merged PRs (30d)
52

Description

## Summary

PR #8772 added `--use-buildkit` to shell out to the `docker`/`finch` CLI instead of using the Python Docker SDK. This is great for CI setups with a remote BuildKit endpoint (no local `dockerd`). However, `ContainerClientFactory.create_client()` still runs unconditionally and performs an Engine-API socket ping via the Docker SDK **before** `--use-buildkit` is evaluated. If that ping fails, the build exits before the BuildKit code path is reached — so `--use-buildkit` is effectively unreachable for its main target use case.

Related: #8649, whose maintainer comment foreshadowed this:

> We are working on supporting BuildKit by allowing the CLI to be called directly when a parameter `--use-buildkit` is provided, meaning soon enough using the CLI instead of the SDK would be possible.

The SDK call is still the first thing that runs.

## Environment

- SAM CLI 1.159.1 (post-#8772)
- Python 3.11.10, Linux (container), x86_64
- `docker` CLI 29.4.2 (**no local dockerd**), buildx 0.33.0 with `remote` driver → `tcp://localhost:1234`
- `moby/buildkit:v0.29.0` sidecar on `tcp://localhost:1234`

## Reproducer

1. Container with `docker` CLI + buildx, no `dockerd`.
2. `buildkitd` sidecar on `tcp://localhost:1234`.
3. `docker buildx create --name remote --driver remote tcp://localhost:1234 && docker buildx use remote`
4. `export DOCKER_HOST=tcp://localhost:1234` (required so `docker buildx` picks up the endpoint).
5. Minimal template with one `PackageType: Image` function + trivial Dockerfile.
6. `sam build --use-buildkit` (or `--no-use-container`, or neither).

Direct `docker buildx build ...` against the same endpoint succeeds.

## Expected

With `--use-buildkit`, SAM shells out to `docker` (or `finch`). `docker buildx` handles the remote endpoint natively. Build succeeds.

## Actual

```
Error: Running AWS SAM projects locally requires a container runtime.
Do you have Docker or Finch installed and running?
```

Debug log (`sam build --use-buildkit --debug`):

```
ContainerClientFactory.create_client() called
Trying Docker client creation
Creating Docker container client from environment variable.
Fall back docker api version to 1.44: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
Docker daemon availability check failed with fallback: ('Connection aborted.', ...)
Docker client not created, trying creating Finch client.
Creating Finch container client with base_url=unix:///var/run/finch.sock
Finch daemon availability check failed: ('Connection aborted.', FileNotFoundError(2, ...))
No container runtime available
```

Telemetry: `containerEngine: tcp-local`, `exitReason: ContainerNotReachableException`.

## Suggested fix

When `--use-buildkit` is set (or `use_buildkit = true` in `samconfig.toml`), skip the SDK-based Engine-API ping. Delegate all runtime availability checks to the CLI invocation itself — it will fail fast with a clearer error if the tooling isn't there.

Alternatively: if `--use-buildkit` implies "I have `docker`/`finch` on PATH," check for the CLI binary instead of pinging the Engine API socket.

## Workaround

Run a `docker:dind` sidecar in the pod, point `DOCKER_HOST=tcp://localhost:2375` only for `sam build`. Works, but duplicates the image-build stack (privileged DinD *plus* the existing BuildKit) just to satisfy the pre-check.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.