BuildOp should also support frontends
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 10.3k
- Forks
- 1.5k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 48
Description
Goal
There is an experimental BuildOp that currently implements nested LLB builds, but it was also intended to also implement nested frontend builds. This issue tracks the work on completing that work. I propose a UX that looks something like:
// Definition-based
var build llb.State // Some image that has `build.go` that generates a LLB to stdout
stWithDef := build.Run(llb.Shlex("sh -c \"go run /build.go > /out/buildkit.llb.definition\""))
st := llb.Build(stWithDef) // Represents a state after `buildkit.llb.definition` has been solved.
// Frontend-based
stWithFrontend := llb.Image("docker.io/library/dockerfile:latest")
st := llb.Gateway(stWithFrontend) // Represents a state after the `dockerfile` frontend has been solved.
Background
Currently, there is an experimental BuildOp that allows for nested LLB builds. The current usage of llbbuild.Build is as a llb.StateOption: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/examples/nested-llb/main.go#L24
llbbuild.Build takes the calling llb.State as an source input and refers it as an input edge: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/client/llb/llbbuild/llbbuild.go#L94-L96
This becomes part of the operation's marshalled protobuf definition: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/client/llb/llbbuild/llbbuild.go#L75-L80
Once this goes over the wire, and the solver is solving the graph that includes the BuildOp, the worker will eventually execute the operation. The job responsible for executing the build will initialize the edge with a dep wrapper over the vertex's inputs:
https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/edge.go#L745-L747
Which is eventually passed into (Op).Exec(context.Context, []solver.Result):
https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/edge.go#L866-L870
The result of the llb.State source input is a rootfs that has a LLB definition somewhere in its filesystem (default is buildkit.llb.definition). The BuildOp exec implementation mounts this rootfs to a temporary directory:
https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/llbsolver/ops/build.go#L89-L92
Reads a definition from it and issues a solve request:
https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/llbsolver/ops/build.go#L115-L126
Design
In the protobuf definition for message BuildOp, a field Builder already exists for the purpose of nested frontend builds: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/pb/ops.proto#L155
For definition-based solves, this is set to -1, indicating there is no builder for this op: https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/solver/pb/const.go#L18-L19
However, we can indicate which one of the vertex's inputs is actually one that should be run as a frontend. We will need to return it as part of the input slice, as well as marshal into the BuildOp's input definition. Then on the solver side, we will have access to the rootfs of the frontend image. Using the same pattern as the gateway frontend, we can set up a temporary LLBBridgeForwarder and run the frontend image with its stdio attached to the forwarder.
Questions
-
In order to run the frontend image, the gateway frontend resolves the image config to get the entrypoint, working dir, and environment. However, for constructing the
BuildOp, we only have access to thellb.Output, which could be allb.Imagechained with some other ops. Where would we get thespecs.Image? How do we access the underlying image config? -
We want to refactor the code in the gateway frontend so it can shared with the
Execimplementation forBuildOp. But in order to create aLLBBridgeForwarder, it requiresfrontend.WorkerInfosthat we don't have access to. It seems to be only used for thePing, but I'm not sure what is that for:
https://github.com/moby/buildkit/blob/1dfd864d22006f1ae4f698110dab578c1ec2730b/frontend/gateway/gateway.go#L606-L616 -
In the solver side, we have access to three interfaces that can execute the container. The
frontend.FrontendLLBBridge,worker.Worker, andexecutor.Executor. We're already at the worker, so should we still go through the bridge again? (Perhaps for frontend's constraints?) If not, I'm assuming we should useworker.Workerbecause it'll set up an active rootfs for us.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading client/llb/llbbuild/llbbuild.go, solver/llbsolver/ops/build.go, solver/pb/ops.proto, and solver/pb/const.go, then trace the existing gateway frontend in frontend/gateway/gateway.go. Compare the BuildOp execution path with the solver edge handling and determine how the three design questions affect the proposed nested frontend API; done means nested frontend builds are supported alongside definition-based builds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- backend, build-system, devtools
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100