Allowing http_file to fetch build_file for http_archive
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
Hi awesome Bazel team,
Is there any way we could allow BUILD files for http_archives to be fetched from a remote source (for example, with http_file)? (Or if I'm wrong, and there already is such a way, could you point me in the right direction?)
The problem I'm trying to solve: Allowing for easy reuse of BUILD files for http_archives.
Here's a concrete example from a bunch of Google projects that illustrates the problem this would fix:
Many Google projects depend on Eigen, and currently, it looks like each is hand-copying over and maintaining eigen.BUILD file needed to make it buildable with Bazel.
- See for example the very good, up-to-date original in TensorFlow here: https://github.com/tensorflow/tensorflow/blob/master/third_party/eigen.BUILD
- Then it looks like Magenta copied over an old, less good one from the main TensorFlow a while back: https://github.com/tensorflow/magenta/blob/master/magenta/music/alignment/eigen.BUILD
- And Ceres wishes could depend on TensorFlow's: https://ceres-solver.googlesource.com/ceres-solver/+/master/WORKSPACE
(And a bunch more)
What I (and I think they) wish you could do is just depend on an existing BUILD file written for Eigen. Something like:
```
# Use Eigen BUILD file from TensorFlow (or wherever--some central place so everyone isn't rolling their own)
http_file(
name = "eigen.BUILD",
urls = ["https://raw.githubusercontent.com/tensorflow/tensorflow/v2.1.0/third_party/eigen.BUILD"],
sha256 = "c8805ce048e79b788c8a9b5ed853c4a864dbd88d9c7b395e34adebba7293ad75",
)
# And apply it to Eigen
http_archive(
name = "eigen",
build_file = "@eigen.BUILD//file:file",
url = "https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.zip",
sha256 = "e09b89aae054e9778ee3f606192ee76d645eec82c402c01c648b1fe46b6b9857"
)
```
But this fails because http_archive insists on a regular file:
```
Not a regular file: /external/eigen.BUILD/file/file
```
But it seems like there are ways this could be made to work with remote files, and solve this more cleanly.
Thanks so much,
Chris
P.S. You can, I think, hack around this like rules_boost does, by grabbing the whole repo and then getting the file target out of that and wrapping everything in another layer of indirection. But that's pretty ugly, given you'd have to, e.g., grab the whole tensorflow repo in this case.
Contributor guide
Research direction
No implementation file or test is named. Start by tracing the existing http_file and http_archive handling and reproduce the reported “Not a regular file” error with the example; done means a remotely fetched BUILD file can be supplied to http_archive without the workaround described.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100