envoyproxy / envoyproxy/toolshed

compile: llvm_toolchain_alias must resolve/symlink host-arch minimal LLVM without cross-extension labels or duplicate downloads

Open
#4,929 0 comments 1 reaction 2 assignees Claimed by @phlax View on GitHub
Dominant language
Python
Stars
12
Forks
24
Avg merge
6h 37m
Merged PRs (30d)
92

Description

## Summary

The `llvm_toolchain_alias_extension` (in `bazel/compile/extensions.bzl`, backed by the `llvm_toolchain_alias` repository rule in `bazel/compile/llvm_minimal.bzl`) still does **not** work when `envoy_toolshed` is consumed as an **external** bzlmod module (e.g. from Envoy). PR #4923 (issue #4926) landed on `main` — it did **not break anything**, but it also did **not fix** the external case.

We need a fix that:
1. Works when toolshed is consumed **externally** (Envoy is the root module), not just in toolshed's own build.
2. Does **NOT** duplicate the LLVM download/extraction. The minimal LLVM artifacts are large; avoiding duplicated multi-GB extraction is the entire reason this minimal/alias machinery exists. The alias repo MUST **symlink** into the already-extracted host-arch `llvm_minimal_*` repo, not download/extract its own second copy.

## What was tried and why each failed

All prior failures share one root cause: a `Label("@llvm_minimal_linux_x64//...")` referenced by **apparent name** cannot be resolved from within the `llvm_toolchain_alias_extension`'s visibility namespace when toolshed is an external module.

1. **Bare apparent-name Label inside the repo rule** — `ctx.path(Label("@{}//:BUILD.bazel".format(minimal_repo)))`:
```
Error in path: ... No repository visible as '@llvm_minimal_linux_x64' from repository '@@envoy_toolshed+'
```
2. **Passing `Label("@llvm_minimal_...//:BUILD.bazel")` from `extensions.bzl` as `attr.label` inputs** — fails at extension-eval time:
```
Error in repository_rule: no repository visible as '@llvm_minimal_linux_x64' in the extension '...%llvm_toolchain_alias_extension', but referenced by label '@llvm_minimal_linux_x64//:BUILD.bazel' in attribute 'minimal_linux_x64' ...
```
3. **CURRENT STATE ON MAIN (PR #4923):** `_llvm_toolchain_alias_ext_impl` calls `setup_llvm_minimal()` itself, then passes `Label("@llvm_minimal_...//:BUILD.bazel")` as mandatory `attr.label`s to `llvm_toolchain_alias`. This resolves in **toolshed's own** `MODULE.bazel.lock` (root module `use_repo`s the minimal repos, giving the apparent-name mapping) but **STILL FAILS EXTERNALLY** with the exact same error as #2:
```
ERROR: .../envoy_toolshed+/compile/extensions.bzl:202:25: ...
Error in repository_rule: no repository visible as '@llvm_minimal_linux_x64' in the extension '@@envoy_toolshed+//compile:extensions.bzl%llvm_toolchain_alias_extension', but referenced by label '@llvm_minimal_linux_x64//:BUILD.bazel' in attribute 'minimal_linux_x64' of llvm_toolchain_alias 'llvm_toolchain_llvm'.
```
Root cause: `setup_llvm_minimal()` calls the `llvm_minimal_repo` repo **rule**, which merely *queues* the repos — it does NOT register an apparent-name mapping (`@llvm_minimal_linux_x64`) usable by `Label()` calls in that extension. Creating the repo in the same extension is not sufficient to reference it by apparent name.

### Dead ends to NOT re-attempt
- **Do not** pass a `Label()` or repo-name string from Envoy's `MODULE.bazel`. You cannot construct a `Label()` in `MODULE.bazel` at all; and a plain string like `"@llvm_minimal_linux_x64"` passed via an extension tag resolves in the **consumer's** repo-mapping and is then re-interpreted in the extension's namespace — the same mismatch that keeps failing.
- **Do not** move the alias into an Envoy-side extension. It would resolve (Envoy's root `use_repo` provides the mapping) but drags toolshed-internal LLVM plumbing into every consumer's MODULE.bazel — explicitly not wanted.
- **Do not** duplicate the download/extraction into the alias repo (naive "Option B"). The artifacts are huge; a second extraction defeats the purpose of this machinery. (See constraints below.)

## Required outcome

The alias repo `@llvm_toolchain_llvm` must, on the host platform:
- Locate the **already-extracted** host-arch `llvm_minimal_*` repo's root directory.
- Populate `bin/`, `include/`, `lib/` by **per-child symlinks** into that existing tree (reuse `_ensure_repo_dir` / `_symlink_dir_children`; per-child, NOT whole-directory, because Bazel does not follow a symlinked package dir when sourcing individual filegroup inputs — see existing docstring / the `missing input file '...//:bin/llvm-nm'` failure mode).
- Write its own `BUILD.bazel` from `LLVM_MINIMAL_LLVM_REPO_BUILD` via `ctx.file(...)` (do NOT symlink the minimal repo's BUILD.bazel).
- NOT download or extract a second copy of the LLVM artifact.

And it must do the above **without** referencing the minimal repos by apparent-name `Label`, so it resolves in an external consumer's build.

## Suggested approach: make the label resolvable via canonical repo mapping (not apparent name)

The reliable way to reference another repo from an extension-created repo rule, externally, is to ensure the label carries a **canonical** repo identity via the extension's recorded repo mapping — not a bare apparent name that must be resolved in a foreign namespace.

Concretely, investigate these (in rough order of preference), and pick whichever actually resolves in an EXTERNAL build:

1. **Single-extension ownership + `ctx.path` on the extension-owned repo via canonical label.** Since PR #4923 already has `llvm_toolchain_alias_extension` call `setup_llvm_minimal()` (so the alias extension owns its own copy of the `llvm_minimal_*` repos), reference those repos by their **canonical** label rather than apparent name. Options to obtain a canonical, resolvable reference:
- Have `setup_llvm_minimal()` (or a variant) **return the created repo names**, and construct the label using the module extension's own repo namespace so the recorded repo mapping includes it. The alias repo rule then does `ctx.path().dirname` and symlinks per-child from there — NO second download, because it symlinks into the copy this extension already created/extracted.
- Note: PR #4923's lockfile shows the alias extension already generates its own `llvm_minimal_*` repos AND `recordedRepoMappingEntries` for them in toolshed's build. The gap is that the `Label("@llvm_minimal_...")` in `extensions.bzl` uses the **apparent** name, which isn't mapped externally. Determine why the recorded mapping isn't being applied for the external consumer and fix the reference so it uses the mapped/canonical form.

2. **Have the alias repo rule read the platform + minimal-repo location from data, then `ctx.path` a canonical label** passed as a proper `attr.label` whose value is the **extension-owned** repo (created in the same extension), ensuring the attr value is a canonical label object (not a re-stringified apparent name).

The key acceptance test for whichever mechanism is chosen: it MUST resolve when `envoy_toolshed` is an **external** module and MUST symlink (single extraction, no duplicate download).

## Constraints (hard requirements)
- **No duplicate LLVM download/extraction.** Exactly one extraction of the host-arch artifact; the alias repo symlinks into it. Verify by confirming the alias repo contains symlinks into the minimal repo's extracted tree, not a second downloaded archive.
- **Per-child symlinks** for `bin/`/`include/`/`lib/` (reuse `_ensure_repo_dir`); alias writes its own `BUILD.bazel` from `LLVM_MINIMAL_LLVM_REPO_BUILD`.
- **Public API unchanged:** still `llvm_toolchain_alias_extension`, still generates repo `llvm_toolchain_llvm` exposing the same filegroup targets (`objcopy`, `clang`, `ld`, `nm`, `ar`, `strip`, `symbolizer`, ...). External consumers (Envoy) `use_repo(llvm_toolchain_alias_ext, "llvm_toolchain_llvm")` unchanged.
- Do NOT change `LLVM_MINIMAL_BINS` or the `LLVM_MINIMAL_LLVM_REPO_BUILD` filegroup schema.
- `fail()` still fires for unsupported host platforms (only linux/x86_64, linux/aarch64, macos/arm64 supported).
- WORKSPACE path (`bazel/WORKSPACE`) must keep working; adjust its `llvm_toolchain_alias(...)` invocation to match the final rule signature.
- `bazel/MODULE.bazel` / `bazel/MODULE.bazel.lock` may need updating/regenerating.

## Files in scope
- `bazel/compile/extensions.bzl`
- `bazel/compile/llvm_minimal.bzl`
- `bazel/WORKSPACE`
- `bazel/MODULE.bazel`, `bazel/MODULE.bazel.lock` (as needed)

## Acceptance criteria — MUST verify with an EXTERNAL consumer, not just toolshed's own build

1. **External build passes.** With toolshed consumed as an external bzlmod module where the root module `use_extension`+`use_repo`s `llvm_toolchain_alias_extension` → `llvm_toolchain_llvm` (as Envoy does, via local_path_override), `llvm_toolchain_alias_extension` evaluates with NO "no repository visible as '@llvm_minimal_linux_x64'" error. (Toolshed's own build passing is necessary but NOT sufficient — the previous fix passed there and still failed externally.)
2. **Real execution.** A real build/execution consuming a tool from `@llvm_toolchain_llvm` (e.g. `@llvm_toolchain_llvm//:objcopy`) succeeds — per-child symlinks stage correctly, no "missing input file '...//:bin/llvm-...'" error.
3. **No duplicate extraction.** The alias repo symlinks into the single already-extracted host-arch minimal repo; there is NOT a second download/extraction of the multi-GB LLVM artifact.
4. Public API/target names unchanged; no duplicate helpers / dead code; `fail()` still fires for unsupported platforms.

## Context / history
- #4926 → PR #4923 landed on `main`: correct direction (single-extension ownership, per-child symlinks restored, dead code removed, WORKSPACE updated) and did not break anything, but does not resolve externally (see failure #3 above).
- Downstream Envoy bzlmod branch consumes toolshed via `local_path_override` and reproduces the external failure cleanly — use that shape to validate.
- The reporter independently concluded (and this issue confirms): collapsing the redundant two-repo indirection is desirable, BUT it must not cause a duplicate extraction — symlinking into the single extracted copy is mandatory.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.