bazel-contrib / bazel-contrib/bazel-lib
Add mirror fallbacks for bazel_lib toolchain binary downloads
- Dominant language
- Starlark
- Stars
- 182
- Forks
- 134
- Avg merge
- 1d 46m
- Merged PRs (30d)
- 1
Description
## Summary
`bazel_lib`'s toolchain repository rules download prebuilt binaries from
GitHub releases via a single hardcoded URL, with no `mirror.bazel.build` /
`urls`-list fallback. A GitHub outage (or unauthenticated-request rejection)
breaks any cold `--repository_cache` build that uses these toolchains.
## Evidence
`lib/private/coreutils_toolchain.bzl` (same pattern in 3.2.2, 3.7.1 and main):
```python
# line ~263
url = "https://github.com/uutils/coreutils/releases/download/{}/{}".format(
rctx.attr.version, filename)
rctx.download_and_extract(
url = url, # single URL, no fallback
integrity = COREUTILS_VERSIONS[rctx.attr.version][platform]["sha256"],
)
```
- Single `url`, not a `urls` list; no `mirror.bazel.build` entry.
- `integrity` is present, so a mirror would be trustless (Bazel verifies bytes).
- `rctx.download_and_extract` goes through Bazel's downloader, so
`--downloader_config` `rewrite`/`block` rules apply.
## Affected downloads
- `coreutils` (uutils/coreutils) — in `bazel_lib`.
- `tar`, `jq`, `yq` — split into their own modules (`tar.bzl`, `jq.bzl`,
`yq.bzl`) in 3.0; same single-URL github download pattern.
- Consumers: e.g. `rules_js` uses coreutils for `chmod`/`mkdir`/`cp` on the
npm fetch path (`npm/private/npm_import.bzl`, `utils.bzl`,
`npm_translate_lock_state.bzl`), so every npm extract needs it.
## Contrast: `rules_java`
`remotejdk` downloads include `mirror.bazel.build` URLs; these `bazel_lib`
toolchain downloads do not.
## Reproduction
```bash
cat > /tmp/block-github.cfg <<'EOF'
block github.com
block codeload.github.com
block objects.githubusercontent.com
block release-assets.githubusercontent.com
block raw.githubusercontent.com
EOF
# A target that pulls a bazel_lib toolchain, cold cache, github blocked.
# For example, in Gerrit (a rules_js consumer):
rm -rf /tmp/ob /tmp/repo
bazelisk --output_base=/tmp/ob fetch --repository_cache=/tmp/repo \
--downloader_config=/tmp/block-github.cfg \
//polygerrit-ui/app:node_modules/resemblejs/dir
# -> Error downloading [https://github.com/uutils/coreutils/releases/download/...]:
# Configured URL rewriter blocked all URLs
```
## Request
Give the toolchain downloads (coreutils, tar, jq, yq) a fallback URL:
accept a `urls` list / expose a mirror-URL attribute, and optionally add a
`mirror.bazel.build` URL if Bazel infra can host the artifacts. `integrity`
is already set, so a fallback is verified and safe.
## Environment
- `bazel_lib` 3.2.2 / `aspect_bazel_lib` 2.22.5 (same single-URL pattern in
3.7.1 and main)
- Bazel 9.1.0
- consumer: `aspect_rules_js`
## References
- Bazel #21687 — [Mirror] bazel-lib copy_to_directory-linux_amd64:
https://github.com/bazelbuild/bazel/issues/21687
- bazel-lib #51 — Mirror releases to bcr (closed; scoped to bazel-lib's own
releases, not third-party coreutils):
https://github.com/bazel-contrib/bazel-lib/issues/51
- bazel-lib #671 — register_coreutils_toolchains version attribute:
https://github.com/bazel-contrib/bazel-lib/issues/671
- Source: https://github.com/bazel-contrib/bazel-lib/blob/main/lib/private/coreutils_toolchain.bzl
- Bazel downloader config (rewrite/block):
https://bazel.build/reference/command-line-reference#flag--downloader_config
Contributor guide
Research direction
Start with lib/private/coreutils_toolchain.bzl and compare the download patterns in tar.bzl, jq.bzl, and yq.bzl. Run the blocked-GitHub reproduction with a cold repository cache to confirm the failure. Done means the toolchain downloads have a verified fallback URL path and the affected build succeeds when GitHub URLs are blocked.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100