moby / moby/buildkit

BuildOp should also support frontends

Open
#1,289 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/llb kind/enhancement
Dominant language
Go
Stars
10.3k
Forks
1.5k
Avg merge
1d 23h
Merged PRs (30d)
48

Description

Goal

There is an experimental BuildOp that currently implements nested LLB builds, but it was also intended to also implement nested frontend builds. This issue tracks the work on completing that work. I propose a UX that looks something like:

// Definition-based
var build llb.State // Some image that has `build.go` that generates a LLB to stdout
stWithDef := build.Run(llb.Shlex("sh -c \"go run /build.go > /out/buildkit.llb.definition\""))
st := llb.Build(stWithDef) // Represents a state after `buildkit.llb.definition` has been solved.

// Frontend-based
stWithFrontend := llb.Image("docker.io/library/dockerfile:latest")
st := llb.Gateway(stWithFrontend) // Represents a state after the `dockerfile` frontend has been solved.

Background

Currently, there is an experimental BuildOp that allows for nested LLB builds. The current usage of llbbuild.Build is as a llb.StateOption: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/examples/nested-llb/main.go#L24

llbbuild.Build takes the calling llb.State as an source input and refers it as an input edge: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/client/llb/llbbuild/llbbuild.go#L94-L96

This becomes part of the operation's marshalled protobuf definition: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/client/llb/llbbuild/llbbuild.go#L75-L80

Once this goes over the wire, and the solver is solving the graph that includes the BuildOp, the worker will eventually execute the operation. The job responsible for executing the build will initialize the edge with a dep wrapper over the vertex's inputs:
https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/edge.go#L745-L747

Which is eventually passed into (Op).Exec(context.Context, []solver.Result):
https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/edge.go#L866-L870

The result of the llb.State source input is a rootfs that has a LLB definition somewhere in its filesystem (default is buildkit.llb.definition). The BuildOp exec implementation mounts this rootfs to a temporary directory:
https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/llbsolver/ops/build.go#L89-L92

Reads a definition from it and issues a solve request:
https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/llbsolver/ops/build.go#L115-L126

Design

In the protobuf definition for message BuildOp, a field Builder already exists for the purpose of nested frontend builds: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/pb/ops.proto#L155

For definition-based solves, this is set to -1, indicating there is no builder for this op: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/pb/const.go#L18-L19

However, we can indicate which one of the vertex's inputs is actually one that should be run as a frontend. We will need to return it as part of the input slice, as well as marshal into the BuildOp's input definition. Then on the solver side, we will have access to the rootfs of the frontend image. Using the same pattern as the gateway frontend, we can set up a temporary LLBBridgeForwarder and run the frontend image with its stdio attached to the forwarder.


Questions

  1. In order to run the frontend image, the gateway frontend resolves the image config to get the entrypoint, working dir, and environment. However, for constructing the BuildOp, we only have access to the llb.Output, which could be a llb.Image chained with some other ops. Where would we get the specs.Image? How do we access the underlying image config?

  2. We want to refactor the code in the gateway frontend so it can shared with the Exec implementation for BuildOp. But in order to create a LLBBridgeForwarder, it requires frontend.WorkerInfos that we don't have access to. It seems to be only used for the Ping, but I'm not sure what is that for:
    https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/frontend/gateway/gateway.go#L606-L616

  3. In the solver side, we have access to three interfaces that can execute the container. The frontend.FrontendLLBBridge, worker.Worker, and executor.Executor. We're already at the worker, so should we still go through the bridge again? (Perhaps for frontend's constraints?) If not, I'm assuming we should use worker.Worker because it'll set up an active rootfs for us.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading client/llb/llbbuild/llbbuild.go, solver/llbsolver/ops/build.go, solver/pb/ops.proto, and solver/pb/const.go, then trace the existing gateway frontend in frontend/gateway/gateway.go. Compare the BuildOp execution path with the solver edge handling and determine how the three design questions affect the proposed nested frontend API; done means nested frontend builds are supported alongside definition-based builds.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
backend, build-system, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.