envoyproxy / envoyproxy/toolshed

bazel/ci: Restructure matrices — one bzlmod build, everything else tests prebuilt artifacts

Open
#5,083 0 comments 1 reaction 2 assignees Claimed by @phlax View on GitHub
Dominant language
Python
Stars
12
Forks
24
Avg merge
6h 37m
Merged PRs (30d)
92

Description

## Problem

`bazel.yml` currently runs six `build` legs and eleven `test` legs, all routed through `_bazel.yml`. Five of the six build legs (`workspace`, `workspace-gcc`, `bzlmod-gcc`, `workspace-macos`, `bzlmod-macos`) set `upload: false` and `artifacts: ''` — they build outputs that are immediately discarded. They exist to prove a configuration still compiles, but they pay full build cost to do it, and they duplicate work the one uploading leg (`bzlmod`) already does.

This is the dominant cost in the pipeline and most of it produces nothing.

## Target shape

**One build job.** `bzlmod`, on Linux, building all artifacts for all arches and toolchains. This is the only leg that compiles anything for keeps and the only leg that uploads. It probably does not need to be a matrix at all — a single job with a fixed target list.

**Everything else becomes a test.** Where a leg currently exists as a `build`, it should instead *test* whatever it was building, rather than rebuilding it.

**Workspace legs test only the prebuilt things.** Workspace mode should not be testing the building of things it will never build. The packaging rules (`//v8:wee8_packages`, `//compile:llvm_minimal_packages`, `//:macos_packages`) depend on bzlmod module extensions and are not defined under WORKSPACE at all — they are already excluded. The workspace legs should be scoped to asserting the workspace path still resolves and its own tests pass, not to reproducing build outputs.

**bzlmod tests should consume the pipeline's own artifacts.** Ideally the bzlmod test legs test against the artifacts produced by the build job in the same run — i.e. the unpublished ones — rather than rebuilding everything to test it. This is the part that needs the most design thought (see below).

## Open questions

### How do test legs consume in-run artifacts?

The build job uploads to the `bazel-artifacts` artifact. Test legs could download it and point the toolchain repos at local paths, which is roughly what `verify-artifacts` already does — it downloads the artifact and runs `.github/verify/artifacts/*.sh` against it. That job is arguably the model for what the other test legs should become.

The friction is that Bazel repo rules want URLs and integrity hashes, so feeding them a local file means either `--override_repository`, a `file://` URL with the hash recomputed at runtime, or a local registry override. Which of these is workable depends on how the LLVM and v8 repos are declared — worth settling before committing to a shape.

### Which legs become `--nobuild` vs. real tests?

For legs that exist purely to prove a configuration analyses cleanly (the gcc legs, arguably the workspace legs), `bazel build --nobuild` does analysis only — it catches broken deps and toolchain resolution failures, which is the actual failure mode, without doing the expensive work. For legs with genuine test targets, run the tests.

Worth deciding this per leg rather than applying one rule.

### macOS

The macOS legs currently build under both `workspace` and `bzlmod` with `upload: false`. The darwin artifacts are produced by the Linux build job (`libcxx-llvm22.1.8-darwin-aarch64.tar.xz`, `//:macos_packages`), so the macOS legs are duplicating that. What they can uniquely establish is that the produced darwin artifacts actually *work* on darwin — which is a test against the uploaded artifact, not a build.

## Interaction with the repository cache work

This reshapes the consumer side of envoyproxy/toolshed#5082. With one build leg, the split becomes a clean writer/reader relationship: the build populates the repository cache with the heavy pinned archives (LLVM ×3, v8 source), and the test legs restore only `rest`. That is simpler than the symmetric five-bucket design sized for the current matrix, and it makes the "do test legs need the heavy buckets?" question mostly answer itself — if tests consume prebuilt artifacts, they should not be fetching toolchain sources at all.

Landing the cache work first is still the right order (it is independent of matrix shape and speeds everything up meanwhile), but the bucket partitioning should probably wait until this restructure settles.

## Constraints

- All legs continue to route through `.github/workflows/_bazel.yml`.
- Follow the conventions from the container work: no `${{ }}` expressions inside `run:` blocks, env vars only, non-trivial logic in checked-in scripts.
- With the matrix gone from `build`, the `name` input to `_bazel.yml` (added only to scope per-leg concurrency groups) can likely be dropped along with the reusable-workflow `concurrency:` block, which is redundant now that parent cancellation propagates correctly.
- `verify-artifacts` must keep asserting that uploaded artifacts contain what their filenames claim — toolchain resolution can silently fall back and produce a green build with a mislabelled tarball.

## Depends on

The `ci-container` branch landing.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.