envoyproxy / envoyproxy/toolshed
compile: llvm_toolchain_alias must resolve/symlink host-arch minimal LLVM without cross-extension labels or duplicate downloads
- Dominant language
- Python
- Stars
- 12
- Forks
- 24
- Avg merge
- 6h 37m
- Merged PRs (30d)
- 92
Description
## Summary
The `llvm_toolchain_alias_extension` (in `bazel/compile/extensions.bzl`, backed by the `llvm_toolchain_alias` repository rule in `bazel/compile/llvm_minimal.bzl`) still does **not** work when `envoy_toolshed` is consumed as an **external** bzlmod module (e.g. from Envoy). PR #4923 (issue #4926) landed on `main` — it did **not break anything**, but it also did **not fix** the external case.
We need a fix that:
1. Works when toolshed is consumed **externally** (Envoy is the root module), not just in toolshed's own build.
2. Does **NOT** duplicate the LLVM download/extraction. The minimal LLVM artifacts are large; avoiding duplicated multi-GB extraction is the entire reason this minimal/alias machinery exists. The alias repo MUST **symlink** into the already-extracted host-arch `llvm_minimal_*` repo, not download/extract its own second copy.
## What was tried and why each failed
All prior failures share one root cause: a `Label("@llvm_minimal_linux_x64//...")` referenced by **apparent name** cannot be resolved from within the `llvm_toolchain_alias_extension`'s visibility namespace when toolshed is an external module.
1. **Bare apparent-name Label inside the repo rule** — `ctx.path(Label("@{}//:BUILD.bazel".format(minimal_repo)))`:
```
Error in path: ... No repository visible as '@llvm_minimal_linux_x64' from repository '@@envoy_toolshed+'
```
2. **Passing `Label("@llvm_minimal_...//:BUILD.bazel")` from `extensions.bzl` as `attr.label` inputs** — fails at extension-eval time:
```
Error in repository_rule: no repository visible as '@llvm_minimal_linux_x64' in the extension '...%llvm_toolchain_alias_extension', but referenced by label '@llvm_minimal_linux_x64//:BUILD.bazel' in attribute 'minimal_linux_x64' ...
```
3. **CURRENT STATE ON MAIN (PR #4923):** `_llvm_toolchain_alias_ext_impl` calls `setup_llvm_minimal()` itself, then passes `Label("@llvm_minimal_...//:BUILD.bazel")` as mandatory `attr.label`s to `llvm_toolchain_alias`. This resolves in **toolshed's own** `MODULE.bazel.lock` (root module `use_repo`s the minimal repos, giving the apparent-name mapping) but **STILL FAILS EXTERNALLY** with the exact same error as #2:
```
ERROR: .../envoy_toolshed+/compile/extensions.bzl:202:25: ...
Error in repository_rule: no repository visible as '@llvm_minimal_linux_x64' in the extension '@@envoy_toolshed+//compile:extensions.bzl%llvm_toolchain_alias_extension', but referenced by label '@llvm_minimal_linux_x64//:BUILD.bazel' in attribute 'minimal_linux_x64' of llvm_toolchain_alias 'llvm_toolchain_llvm'.
```
Root cause: `setup_llvm_minimal()` calls the `llvm_minimal_repo` repo **rule**, which merely *queues* the repos — it does NOT register an apparent-name mapping (`@llvm_minimal_linux_x64`) usable by `Label()` calls in that extension. Creating the repo in the same extension is not sufficient to reference it by apparent name.
### Dead ends to NOT re-attempt
- **Do not** pass a `Label()` or repo-name string from Envoy's `MODULE.bazel`. You cannot construct a `Label()` in `MODULE.bazel` at all; and a plain string like `"@llvm_minimal_linux_x64"` passed via an extension tag resolves in the **consumer's** repo-mapping and is then re-interpreted in the extension's namespace — the same mismatch that keeps failing.
- **Do not** move the alias into an Envoy-side extension. It would resolve (Envoy's root `use_repo` provides the mapping) but drags toolshed-internal LLVM plumbing into every consumer's MODULE.bazel — explicitly not wanted.
- **Do not** duplicate the download/extraction into the alias repo (naive "Option B"). The artifacts are huge; a second extraction defeats the purpose of this machinery. (See constraints below.)
## Required outcome
The alias repo `@llvm_toolchain_llvm` must, on the host platform:
- Locate the **already-extracted** host-arch `llvm_minimal_*` repo's root directory.
- Populate `bin/`, `include/`, `lib/` by **per-child symlinks** into that existing tree (reuse `_ensure_repo_dir` / `_symlink_dir_children`; per-child, NOT whole-directory, because Bazel does not follow a symlinked package dir when sourcing individual filegroup inputs — see existing docstring / the `missing input file '...//:bin/llvm-nm'` failure mode).
- Write its own `BUILD.bazel` from `LLVM_MINIMAL_LLVM_REPO_BUILD` via `ctx.file(...)` (do NOT symlink the minimal repo's BUILD.bazel).
- NOT download or extract a second copy of the LLVM artifact.
And it must do the above **without** referencing the minimal repos by apparent-name `Label`, so it resolves in an external consumer's build.
## Suggested approach: make the label resolvable via canonical repo mapping (not apparent name)
The reliable way to reference another repo from an extension-created repo rule, externally, is to ensure the label carries a **canonical** repo identity via the extension's recorded repo mapping — not a bare apparent name that must be resolved in a foreign namespace.
Concretely, investigate these (in rough order of preference), and pick whichever actually resolves in an EXTERNAL build:
1. **Single-extension ownership + `ctx.path` on the extension-owned repo via canonical label.** Since PR #4923 already has `llvm_toolchain_alias_extension` call `setup_llvm_minimal()` (so the alias extension owns its own copy of the `llvm_minimal_*` repos), reference those repos by their **canonical** label rather than apparent name. Options to obtain a canonical, resolvable reference:
- Have `setup_llvm_minimal()` (or a variant) **return the created repo names**, and construct the label using the module extension's own repo namespace so the recorded repo mapping includes it. The alias repo rule then does `ctx.path().dirname` and symlinks per-child from there — NO second download, because it symlinks into the copy this extension already created/extracted.
- Note: PR #4923's lockfile shows the alias extension already generates its own `llvm_minimal_*` repos AND `recordedRepoMappingEntries` for them in toolshed's build. The gap is that the `Label("@llvm_minimal_...")` in `extensions.bzl` uses the **apparent** name, which isn't mapped externally. Determine why the recorded mapping isn't being applied for the external consumer and fix the reference so it uses the mapped/canonical form.
2. **Have the alias repo rule read the platform + minimal-repo location from data, then `ctx.path` a canonical label** passed as a proper `attr.label` whose value is the **extension-owned** repo (created in the same extension), ensuring the attr value is a canonical label object (not a re-stringified apparent name).
The key acceptance test for whichever mechanism is chosen: it MUST resolve when `envoy_toolshed` is an **external** module and MUST symlink (single extraction, no duplicate download).
## Constraints (hard requirements)
- **No duplicate LLVM download/extraction.** Exactly one extraction of the host-arch artifact; the alias repo symlinks into it. Verify by confirming the alias repo contains symlinks into the minimal repo's extracted tree, not a second downloaded archive.
- **Per-child symlinks** for `bin/`/`include/`/`lib/` (reuse `_ensure_repo_dir`); alias writes its own `BUILD.bazel` from `LLVM_MINIMAL_LLVM_REPO_BUILD`.
- **Public API unchanged:** still `llvm_toolchain_alias_extension`, still generates repo `llvm_toolchain_llvm` exposing the same filegroup targets (`objcopy`, `clang`, `ld`, `nm`, `ar`, `strip`, `symbolizer`, ...). External consumers (Envoy) `use_repo(llvm_toolchain_alias_ext, "llvm_toolchain_llvm")` unchanged.
- Do NOT change `LLVM_MINIMAL_BINS` or the `LLVM_MINIMAL_LLVM_REPO_BUILD` filegroup schema.
- `fail()` still fires for unsupported host platforms (only linux/x86_64, linux/aarch64, macos/arm64 supported).
- WORKSPACE path (`bazel/WORKSPACE`) must keep working; adjust its `llvm_toolchain_alias(...)` invocation to match the final rule signature.
- `bazel/MODULE.bazel` / `bazel/MODULE.bazel.lock` may need updating/regenerating.
## Files in scope
- `bazel/compile/extensions.bzl`
- `bazel/compile/llvm_minimal.bzl`
- `bazel/WORKSPACE`
- `bazel/MODULE.bazel`, `bazel/MODULE.bazel.lock` (as needed)
## Acceptance criteria — MUST verify with an EXTERNAL consumer, not just toolshed's own build
1. **External build passes.** With toolshed consumed as an external bzlmod module where the root module `use_extension`+`use_repo`s `llvm_toolchain_alias_extension` → `llvm_toolchain_llvm` (as Envoy does, via local_path_override), `llvm_toolchain_alias_extension` evaluates with NO "no repository visible as '@llvm_minimal_linux_x64'" error. (Toolshed's own build passing is necessary but NOT sufficient — the previous fix passed there and still failed externally.)
2. **Real execution.** A real build/execution consuming a tool from `@llvm_toolchain_llvm` (e.g. `@llvm_toolchain_llvm//:objcopy`) succeeds — per-child symlinks stage correctly, no "missing input file '...//:bin/llvm-...'" error.
3. **No duplicate extraction.** The alias repo symlinks into the single already-extracted host-arch minimal repo; there is NOT a second download/extraction of the multi-GB LLVM artifact.
4. Public API/target names unchanged; no duplicate helpers / dead code; `fail()` still fires for unsupported platforms.
## Context / history
- #4926 → PR #4923 landed on `main`: correct direction (single-extension ownership, per-child symlinks restored, dead code removed, WORKSPACE updated) and did not break anything, but does not resolve externally (see failure #3 above).
- Downstream Envoy bzlmod branch consumes toolshed via `local_path_override` and reproduces the external failure cleanly — use that shape to validate.
- The reporter independently concluded (and this issue confirms): collapsing the redundant two-repo indirection is desirable, BUT it must not cause a duplicate extraction — symlinking into the single extracted copy is mandatory.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.