bazelbuild / bazelbuild/bazel

Only invalidate repositories when an environment variables changes that they read

Open
#20,951 2 comments 0 reactions 0 assignees View on GitHub
P2 team-ExternalDeps type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the feature request:

Currently, external repositories are dirtied in Skyframe when any variable in `--repo_env` changes.

This means that there is a lot of unnecessary re-hashing to establish that the thus dirtied repositories are in fact up to date. This is because currently every repository depends on `PRECOMPUTED:repo_env`, which is just a bag of every such environment variable.

Ideally, repositories should only depend on the variables that they actually access.

There is some prior art in this area:
* https://github.com/bazelbuild/bazel/commit/c230e39fb225edd206ed0aa07cfcdd8c51589965 adds the ability to gather the accessed environment variables (not strictly necessary, but is a good pleprint)
* https://github.com/bazelbuild/bazel/commit/80a4a1483e38353e9472ed8bfd5261efe1a6e23b makes repository rules not depend on `PRECOMPUTED:repo_env` if they don't read any environment variable. It's very crude, though.

### Which category does this issue belong to?

_No response_

### What underlying problem are you trying to solve with this feature?

_No response_

### Which operating system are you running Bazel on?

_No response_

### What is the output of `bazel info release`?

_No response_

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reading the handling of --repo_env, PRECOMPUTED:repo_env, and external repository dependencies, then compare the prior-art commits linked in the issue. Define how accessed variables are gathered and how repository rules depend only on those variables; done means unrelated --repo_env changes no longer invalidate a repository.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.