coder / coder/internal

Enhance Docker Build Script for Compatibility with Containerd Layer Store

Open
#254 0 comments 0 reactions 0 assignees View on GitHub
help wanted tech-debt
Dominant language
No language data
Stars
3
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Problem Description
The recent introduction of the `containerd` layer store in Depot-hosted runners exposed gaps in our understanding and configuration of Docker’s build process. The feature is working as designed, but it conflicts with our existing workflow when handling [multi-architecture builds](https://docs.docker.com/build/building/multi-platform/) and provenance data.

Specifically:
- Single-architecture images inadvertently include manifest lists with an unexpected "unknown" platform due to Docker's default build provenance metadata.
- The legacy Docker layer store previously stripped this metadata, but the `containerd` store natively supports manifest lists and does not modify the image format.
- Modifying our scripts is necessary to fully support the `containerd` layer store while retaining provenance data and enabling multi-architecture builds.

## Desired Solution
1. **Update `build_docker.sh` to Use `docker buildx`:**
- Incorporate `docker buildx build` to handle [multi-platform builds](https://docs.docker.com/build/building/multi-platform/) efficiently.
- Ensure proper handling of provenance metadata and manifest merging when creating multi-arch images.

2. **Integrate `depot build` as an Optional Workflow:**
- Add a flag to toggle between `docker buildx` and `depot build` for CI environments.
- Use `depot build` for efficient multi-arch builds where possible, falling back to `docker buildx` for broader compatibility.

3. **Enhance CI Workflow:**
- Default to `docker buildx` for local and dev workflows to avoid additional dependencies.
- Enable `depot build` only in CI and set it up dynamically using a GitHub Action.

4. **Optimize for Single-Arch Builds:**
- Modify the script to build single-architecture images efficiently (e.g., during dogfood development) while keeping multi-arch workflows intact in CI.

## Next Steps
- Update `scripts/build_docker.sh` to support both `docker buildx` for local dev and `depot build` for ci.
- Test single-architecture and multi-architecture builds using both approaches to ensure functionality with the `containerd` layer store.

## References
- [Docker Buildx Documentation](https://docs.docker.com/buildx/working-with-buildx/)
- [Docker Provenance Metadata](https://docs.docker.com/build/metadata/attestations/slsa-provenance/)
- [`containerd` image store](https://docs.docker.com/desktop/features/containerd/)
- [Multi-platform builds](https://docs.docker.com/build/building/multi-platform/)
- [Slack Thread](https://codercom.slack.com/archives/C07AL9CP7B2/p1734075806309169)

This issue will help us fully leverage the `containerd` layer store's benefits while maintaining compatibility and development flexibility.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.