Parallelize repository up-to-dateness check
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the feature request:
External repositories need to verify that their file dependencies are still up-to-date. This happens on one thread in `RepositoryFunction.verifyMarkerDataForFiles()`. This means that if a repository has many files to check, we are leaving a lot of performance on the table. It would be nice if we used all the parallelism available (it's mostly checksumming, so it's not obvious whether it's I/O or CPU-bound, but more parallelism helps either way, the question is just how much more)
The obvious way to do this is to have a little thread pool, but we already have many little thread pools (e.g. `TreeArtifactValue.VISITOR_POOL` which does something similar), so maybe some consolidation is in order.
### Which category does this issue belong to?
_No response_
### What underlying problem are you trying to solve with this feature?
Make null builds faster.
### Which operating system are you running Bazel on?
_No response_
### What is the output of `bazel info release`?
_No response_
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?
```text
`7dd569f1b0166cbf651789d91ce192d8bde44267`
```
### Have you found anything relevant by searching the web?
_No response_
### Any other information, logs, or outputs that you want to share?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.