bazelbuild / bazelbuild/bazel

Repository downloader drops netrc/`auth` credentials on same-host HTTP redirects

Open
#30,013 0 comments 0 reactions 0 assignees View on GitHub
help wanted P2 team-ExternalDeps type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 18h
Merged PRs (30d)
75

Description

### Description of the bug:

When `http_file`/`http_archive` downloads a URL that responds with a redirect to another path **on the same host**, Bazel does not send the credentials (from `~/.netrc` or the rule's `auth` attr) on the redirected request. The follow-up request goes out unauthenticated and fails (e.g. `401`). This commonly surfaces with artifact servers/CDNs that normalize a URL and `30x`-redirect to a canonical path that still requires auth.

**Root cause**

Credentials are flattened into an **exact-`URI` map** and looked up per redirect hop, so a
redirect to any URI other than the original matches nothing:

1. [`use_netrc()` in `tools/build_defs/repo/utils.bzl`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/tools/build_defs/repo/utils.bzl#L427)
keys the `auth` dict by the exact request URL
([keying](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/tools/build_defs/repo/utils.bzl#L471-L483)).
2. [`getAuthHeaders()` in `StarlarkBaseExternalContext.java`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/bazel/repository/starlark/StarlarkBaseExternalContext.java#L332)
turns that dict into an `ImmutableMap` keyed by the request
[`URI`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/bazel/repository/starlark/StarlarkBaseExternalContext.java#L348).
3. [`StaticCredentials.getRequestMetadata(uri)`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/authandtls/StaticCredentials.java#L42)
does an exact-key lookup with no host fallback:
[`credentials.getOrDefault(uri, ImmutableMap.of())`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/authandtls/StaticCredentials.java#L45).
4. [`HttpConnectorMultiplexer.getHeaderFunction()`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/bazel/repository/downloader/HttpConnectorMultiplexer.java#L143)
re-runs this lookup for every hop.
5. [`HttpConnector.connect()`](https://github.com/bazelbuild/bazel/blob/ed7c04979e0e620480b451e1e1a87a524349b7b9/src/main/java/com/google/devtools/build/lib/bazel/repository/downloader/HttpConnector.java#L209)
follows the redirect (`url = HttpUtils.getLocation(connection)`) and re-invokes the header
function with the new target URI.

On the redirect, the new target URI is not a key in the map, so `getOrDefault` returns empty
and no `Authorization` header is added to the follow-up request.

### Which category does this issue belong to?

_No response_

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

`server.py` (a host that redirects `/start` → `/protected`, where `/protected` requires Basic auth):
```python
from http.server import BaseHTTPRequestHandler, HTTPServer
class H(BaseHTTPRequestHandler):
def do_GET(self):
if self.path == "/start":
self.send_response(307); self.send_header("Location", "/protected"); self.end_headers()
elif self.path == "/protected" and self.headers.get("Authorization"):
body=b"ok"; self.send_response(200); self.send_header("Content-Length", str(len(body))); self.end_headers(); self.wfile.write(body)
else:
self.send_response(401); self.end_headers()
def log_message(self,*a): pass
HTTPServer(("127.0.0.1",8077),H).serve_forever()
```

`netrc`:
```
machine localhost login u password p
```

MODULE.bazel:
```python
http_file = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")
http_file(
name = "x",
url = "http://localhost:8077/start",
netrc = "netrc",
sha256 = "2689367b205c16ce32ed4200942b8b8b1e262dfc70d9bc9fbc77c49699a4f1df", # "ok"
)
```

```sh
python3 server.py &
bazel build @x//file
```

**Expected:** download succeeds (creds re-applied to the same-host redirect target → `200`).
**Actual:** `GET returned 401 Unauthorized`. The server log shows `/start` arrives with `Authorization`, but `/protected` (the redirect target) arrives **without** it.

### Which operating system are you running Bazel on?

Linux

### What is the output of `bazel info release`?

9.x

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

```text

```

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

_No response_

### Have you found anything relevant by searching the web?

Stripping `Authorization` on a **cross-host** redirect is standard (security). The difference is on **same-host** redirects:
- **curl `--netrc`** / **wget** re-derive credentials from netrc **per host** on every hop → same-host redirect stays authenticated.
- **Python `requests`** only strips `Authorization` when the host changes; same-host redirects keep it.
- **Bazel** keys by exact URI and re-derives nothing, so even a same-host, same-scheme redirect to a different path loses the credentials.

### Any other information, logs, or outputs that you want to share?

Suggested direction: Re-apply the netrc-derived credentials by **host** (at least for same-host, same-scheme redirects), matching curl/wget — Bazel already has host-based `netrcCreds` available but converts them to an exact-URI entry for the original URL before downloading.

Contributor guide

Open the contributing guide

Research direction

Start with use_netrc() in tools/build_defs/repo/utils.bzl, then trace getAuthHeaders() and StaticCredentials.getRequestMetadata() through HttpConnectorMultiplexer.getHeaderFunction() and HttpConnector.connect(). Reproduce the issue with server.py, netrc, and the MODULE.bazel example using bazel build @x//file. Done means credentials are retained for same-host redirects while the download succeeds with the protected response.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
57/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.