bazelbuild / bazelbuild/bazel

http_archive looks for netrc file inside external dependency rather than local file system

Open
#13,662 8 comments 4 reactions 0 assignees View on GitHub
P2 team-ExternalDeps type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 18h
Merged PRs (30d)
75

Description

### Description of the problem / feature request:

As the title says, http_archive appears to have a bug where it looks for netrc inside the external dependency, rather than in the local workspace.

### Feature requests: what underlying problem are you trying to solve with this feature?

The ability to use with http_archive a custom netrc file in the local workspace instead of in my home directory. Current behavior makes it effectively unusable.

### Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

WORKSPACE contents:

```
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
name = "halide-x86-64-osx",
build_file = "//third_party/halide:halide.BUILD",
netrc = "artifactory.netrc",
sha256 = "e83d2efa273c92ba8c14da7ac3fda88a25c8970c2099befa92caee67573d9681",
strip_prefix = "Halide-12.0.1-x86-64-osx",
url = "http://localhost", # Doesn't actually matter - bug manifests before actually parsing this.
)
```

BUILD contents:

```
cc_library(
name = "my_lib", # No sources doesn't matter - bug manifests when trying to resolve the dependency
deps = ["@halide-x86-64-osx//:halide"],
)
```

`artifactory.netrc` can be an empty file - its contents don't matter when reproducing this bug.

Build result:

```
$ bazel build :my_lib
INFO: Repository halide-x86-64-osx instantiated at:
/Users/jiawen/tmp/y/WORKSPACE:3:13: in
Repository rule http_archive defined at:
/private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in
ERROR: An error occurred during the fetch of repository 'halide-x86-64-osx':
Traceback (most recent call last):
File "/private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 109, column 21, in _http_archive_impl
auth = _get_auth(ctx, all_urls)
File "/private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 79, column 27, in _get_auth
netrc = read_netrc(ctx, ctx.attr.netrc)
File "/private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 215, column 24, in read_netrc
contents = ctx.read(filename)
Error in read: java.io.FileNotFoundException: /private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/halide-x86-64-osx/artifactory.netrc (No such file or directory)
ERROR: Error fetching repository: Traceback (most recent call last):
File "/private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 109, column 21, in _http_archive_impl
auth = _get_auth(ctx, all_urls)
File "/private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/bazel_tools/tools/build_defs/repo/http.bzl", line 79, column 27, in _get_auth
netrc = read_netrc(ctx, ctx.attr.netrc)
File "/private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 215, column 24, in read_netrc
contents = ctx.read(filename)
Error in read: java.io.FileNotFoundException: /private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/halide-x86-64-osx/artifactory.netrc (No such file or directory)
ERROR: /Users/jiawen/tmp/y/BUILD:1:11: //:my_lib depends on @halide-x86-64-osx//:halide in repository @halide-x86-64-osx which failed to fetch. no such package '@halide-x86-64-osx//': java.io.FileNotFoundException: /private/var/tmp/_bazel_jiawen/d9402db5073cf8cd36f8900f74d53c1a/external/halide-x86-64-osx/artifactory.netrc (No such file or directory)
ERROR: Analysis of target '//:my_lib' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.229s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (2 packages loaded, 2 targets configured)
```

### What operating system are you running Bazel on?

MacOS Big Sur

### What's the output of `bazel info release`?

release 4.1.0-homebrew

### If `bazel info release` returns "development version" or "(@non-git)", tell us how you built Bazel.

N/A

### What's the output of `git remote get-url origin ; git rev-parse master ; git rev-parse HEAD` ?

(Redacted)

### Have you found anything relevant by searching the web?

No, http_archive with custom netrc files seems to be not broadly used.

### Any other information, logs, or outputs that you want to share?

No - it is clear that the path to the netrc file is somehow read from the wrong context. The implementation is here:

[_get_auth](https://github.com/bazelbuild/bazel/blob/master/tools/build_defs/repo/http.bzl#L79) calls [read_netrc](https://github.com/bazelbuild/bazel/blob/786b418d1fcd1ad1df66237be45682624007ed2e/tools/build_defs/repo/utils.bzl#L247) with a context but I'm not well versed enough in Bazel to fix this.

Contributor guide

Open the contributing guide

Research direction

Start with tools/build_defs/repo/http.bzl, especially _get_auth, and tools/build_defs/repo/utils.bzl at read_netrc. Reproduce the issue with the supplied WORKSPACE, BUILD, and bazel build :my_lib command, then verify that a netrc path such as artifactory.netrc is resolved from the local workspace rather than the external repository directory.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.