Repository downloader drops netrc/`auth` credentials on same-host HTTP redirects
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 75
Description
### Description of the bug:
When `http_file`/`http_archive` downloads a URL that responds with a redirect to another path **on the same host**, Bazel does not send the credentials (from `~/.netrc` or the rule's `auth` attr) on the redirected request. The follow-up request goes out unauthenticated and fails (e.g. `401`). This commonly surfaces with artifact servers/CDNs that normalize a URL and `30x`-redirect to a canonical path that still requires auth.
**Root cause**
Credentials are flattened into an **exact-`URI` map** and looked up per redirect hop, so a
redirect to any URI other than the original matches nothing:
1. [`use_netrc()` in `tools/build_defs/repo/utils.bzl`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/tools/build_defs/repo/utils.bzl#L427)
keys the `auth` dict by the exact request URL
([keying](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/tools/build_defs/repo/utils.bzl#L471-L483)).
2. [`getAuthHeaders()` in `StarlarkBaseExternalContext.java`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/bazel/repository/starlark/StarlarkBaseExternalContext.java#L332)
turns that dict into an `ImmutableMap` keyed by the request
[`URI`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/bazel/repository/starlark/StarlarkBaseExternalContext.java#L348).
3. [`StaticCredentials.getRequestMetadata(uri)`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/authandtls/StaticCredentials.java#L42)
does an exact-key lookup with no host fallback:
[`credentials.getOrDefault(uri, ImmutableMap.of())`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/authandtls/StaticCredentials.java#L45).
4. [`HttpConnectorMultiplexer.getHeaderFunction()`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/bazel/repository/downloader/HttpConnectorMultiplexer.java#L143)
re-runs this lookup for every hop.
5. [`HttpConnector.connect()`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/bazel/repository/downloader/HttpConnector.java#L209)
follows the redirect (`url = HttpUtils.getLocation(connection)`) and re-invokes the header
function with the new target URI.
On the redirect, the new target URI is not a key in the map, so `getOrDefault` returns empty
and no `Authorization` header is added to the follow-up request.
### Which category does this issue belong to?
_No response_
### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
`server.py` (a host that redirects `/start` → `/protected`, where `/protected` requires Basic auth):
```python
from http.server import BaseHTTPRequestHandler, HTTPServer
class H(BaseHTTPRequestHandler):
def do_GET(self):
if self.path == "/start":
self.send_response(307); self.send_header("Location", "/protected"); self.end_headers()
elif self.path == "/protected" and self.headers.get("Authorization"):
body=b"ok"; self.send_response(200); self.send_header("Content-Length", str(len(body))); self.end_headers(); self.wfile.write(body)
else:
self.send_response(401); self.end_headers()
def log_message(self,*a): pass
HTTPServer(("127.0.0.1",8077),H).serve_forever()
```
`netrc`:
```
machine localhost login u password p
```
MODULE.bazel:
```python
http_file = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")
http_file(
name = "x",
url = "http://localhost:8077/start",
netrc = "netrc",
sha256 = "2689367b205c16ce32ed4200942b8b8b1e262dfc70d9bc9fbc77c49699a4f1df", # "ok"
)
```
```sh
python3 server.py &
bazel build @x//file
```
**Expected:** download succeeds (creds re-applied to the same-host redirect target → `200`).
**Actual:** `GET returned 401 Unauthorized`. The server log shows `/start` arrives with `Authorization`, but `/protected` (the redirect target) arrives **without** it.
### Which operating system are you running Bazel on?
Linux
### What is the output of `bazel info release`?
9.x
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse HEAD` ?
```text
```
### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.
_No response_
### Have you found anything relevant by searching the web?
Stripping `Authorization` on a **cross-host** redirect is standard (security). The difference is on **same-host** redirects:
- **curl `--netrc`** / **wget** re-derive credentials from netrc **per host** on every hop → same-host redirect stays authenticated.
- **Python `requests`** only strips `Authorization` when the host changes; same-host redirects keep it.
- **Bazel** keys by exact URI and re-derives nothing, so even a same-host, same-scheme redirect to a different path loses the credentials.
### Any other information, logs, or outputs that you want to share?
Suggested direction: Re-apply the netrc-derived credentials by **host** (at least for same-host, same-scheme redirects), matching curl/wget — Bazel already has host-based `netrcCreds` available but converts them to an exact-URI entry for the original URL before downloading.
Contributor guide
Research direction
Start with use_netrc() in tools/build_defs/repo/utils.bzl, then trace getAuthHeaders() and StaticCredentials.getRequestMetadata() through HttpConnectorMultiplexer.getHeaderFunction() and HttpConnector.connect(). Reproduce the issue with server.py, netrc, and the MODULE.bazel example using bazel build @x//file. Done means credentials are retained for same-host redirects while the download succeeds with the protected response.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 57/100