bazelbuild / bazelbuild/bazel

allow repository_rules to run on stale state

Open
#23,720 7 comments 0 reactions 0 assignees View on GitHub
help wanted P2 team-ExternalDeps type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the feature request:

repository_rules are a way to run arbitrary imperative operations during bazel's loading phase. The default is that bazel clears their output folder before each execution of the repository rule - a good default, but it makes some use cases challenging. I propose adding a parameter to repository_rule: `clear_output_folder_before_fetch` which should be True by default (maintaining current behavior), but could be set to False for use cases that trust their rules to properly handle stale state.

### Which category does this issue belong to?

Rules API

### What underlying problem are you trying to solve with this feature?

I've written a repository rule using yarn to install node_modules/ which we've integrated with our custom js library rules. Yarn itself has a ton of logic to handle incremental updating of the generated node_modules/, so that `yarn install` after a small change to the yarn.lock can take a few seconds rather than a minute or more. Since bazel clears the repository rule output folder, either we end up with quite bad performance, or we need quite a bit of extra complexity to put the outputs in a different folder and symlink them to bazel's output folder for the repository rule, so that we only have to create and rewrite symlinks on each repeat invocation and let yarn take care of the rest, as opposed to starting from scratch; (the symlink hacks cost us a few seconds of execution time - and also have caused us to encounter bugs in yarn - so aren't free either, but much better than no workaround)

Likely other "use a package manager or other build system in a repository rule" use cases have this problem as well.

### Which operating system are you running Bazel on?

linux - ubuntu 22.04

### What is the output of `bazel info release`?

release 6.1.0-1c2df03733503215490047c398d25fe5b5553a0e

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.