GoogleContainerTools / GoogleContainerTools/skaffold

Support additional docker buildkit options

Open
#9,197 0 comments 17 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

Currently BuildKit / buildx support is Skaffold seems to be rudimentary:
When `useBuildKit:true` it simply sets env `DOCKER_BUILDKIT=1`, effectively enabling BuildKit
but not allowing to specify any of the command line options that `docker buildx build` support, for example.
This means that `--cache-from` with, for example `type=registry` can not be used without a custom build command like this one:

```sh
apiVersion: 'skaffold/v3'
kind: 'Config'
metadata:
name: 'buildx-inline-caching'
build:
artifacts:
- image: us-docker.pkg.dev/skaffold-examples/buildx-inline-caching
context: .
custom:
buildCommand: |
echo
echo "Building image '${IMAGE}' for platforms '${PLATFORMS}' from context '${BUILD_CONTEXT}'"
echo "Image is expected to exist remotely: ${PUSH_IMAGE}"
echo
args=()
if [[ "${PUSH_IMAGE}" == true ]]; then
args+=("--push")
elif [[ "${PLATFORMS}" != *","* ]]; then
args+=("--load")
fi
docker buildx build \
--file "${BUILD_CONTEXT}/Dockerfile" \
--tag "${IMAGE}" \
--cache-to "type=inline" \
--cache-from "type=registry,ref=us-docker.pkg.dev/skaffold-examples/buildx-inline-caching:cache" \
--platform "${PLATFORMS}" \
"${args[@]}" \
"${BUILD_CONTEXT}"
```

This is what we currently use.
Skaffold is being invoked with `--tag cache` after a successful build, which re-tags and pushes the image with the `:cache` tag so it can be used as a cache source in subsequent builds.
However this approach doesn't really scale well, having to implement this custom script across potentially hundreds of `skaffold.yaml` files and repositories.
It would be way nicer if the `buildx build` arguments would be supported by Skaffold natively.

As a start, I imagine using the combination of the `useBuildKit` and `cacheFrom` docker builder config options to construct the `--cache-from` option for example and adding a `cacheTo` config option for the docker builder to do the same for the `--cache-to` option.
This way one could leverage the advanced caching mechanisms of BuildKit _through_ Skaffold.
Replacing the call to `docker build` with `docker buildx build` should already be transparent once you installed buildx (it is on my machine), but I need to double check.

Let me know what you think!
I am happy to provide a first implementation, but I could definitely use some helping getting the tests right.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.