sandbox for repository rules
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the problem / feature request:
I would like a way to sandox repository rules to ensure reproducible. Especially, `repository_ctx.execute` (https://docs.bazel.build/versions/master/skylark/lib/repository_ctx.html) should provide a way to list the file that the command can access.
### Feature requests: what underlying problem are you trying to solve with this feature?
I'm working on rules_nixpkgs https://github.com/tweag/rules_nixpkgs which provides repository rules to import pre-built libraries / executable from `nixpkgs`. We use that with great success to import lot of libraries without fully rewriting Bazel rules for them.
However, the current status of the `nixpkgs_package` from rules_nixpkgs is not fully reproducible, see https://github.com/tweag/rules_nixpkgs/issues/74
`nixpkgs_package` executes a command (`nix-build`) which reads a local "nix file" specified as an attribute of the rule. This file may indirectly depend on other files through nix `import` mechanism. Unfortunately bazel will invalidate the cache only if the initial "nix file" changes. To mitigate this issue, I added another argument which list all the indirect dependencies.
Thank to this argument, bazel will invalidate the cache, and recompute the repository rule, if any of the listed indirect dependencies is changed. If we change an indirect dependency which is not listed (by mistake) in the list of indirect dependencies, the cache won't be invalidated. That's a huge **reproducibility** issue.
This is a bug in rules_nixpkgs! But it can be solved with a more hermetic `repository_ctx.execute`. I'm sure we can find other uses case for more hermetic repository rules, for example, users are requesting a way to force rebuild repository rules at each bazel run for similar reasons: https://github.com/bazelbuild/bazel/issues/3041.
I'm thinking about a sandbox similar to the one used in `actions.run` where the list of `inputs` is to be specified. https://docs.bazel.build/versions/master/skylark/lib/actions.html#run
I'm looking after something like that:
```
repository_ctx.execute(["command", "arg1", "arg2"],
inputs = [
"./aFile.nix",
"./anotherFile.zip",
"/a/path/to/a/directory/"
],
)
```
Will will execute the `command` in a sandbox containing only `aFile.nix`, `anotherFile.zip` and all the content of `/a/path/to/a/directory/`.
### What operating system are you running Bazel on?
Linux, but that feature is important on all systems.
### What's the output of `bazel info release`?
```
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 3a28e27f-86f4-4d8e-b202-e3b9592c09f8
release 0.22.0- (@non-git)
```
I'm not aware of any relevant change in newer bazel version.
### Have you found anything relevant by searching the web?
No.
### Any other information, logs, or outputs that you want to share?
Contributor guide
Research direction
Start with the repository_ctx.execute API and compare its requested inputs behavior with the actions.run sandbox documented in the issue. Review the rules_nixpkgs issue for the reproducibility failure and determine how indirect file access and cache invalidation should be specified. Done means repository rules can execute with an explicit sandbox input set and detect changes to accessed dependencies.
Written by the indexing model from the issue text.
Assessment
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100