bazelbuild / bazelbuild/bazel

Actions fetched from remote cache / executed remotely should not count towards parallelism limit

Open
#6,394 26 comments 3 reactions 1 assignee Claimed by @coeuvre View on GitHub
not stale P2 team-Performance type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

When a fairly large application is built using the remote cache, and all the actions are fully cached. Bazel still keeps the parallelism set to the number of cores in the machine. However, most actions are just waiting for network I/O.

With the default settings on an 8-core machine, I get:

```
$ time bazel build //...
## output elided
INFO: Elapsed time: 97.567s, Critical Path: 26.44s
INFO: 2000 processes: 2000 remote cache hit.
INFO: Build completed successfully, 2240 total actions

real 1m38.518s
user 0m0.051s
sys 0m0.061s
```

But if I bump up the number of jobs to a crazy number:

```
$ time bazel build --jobs 2000 //...
## output elided
INFO: Elapsed time: 39.535s, Critical Path: 31.33s
INFO: 2000 processes: 2000 remote cache hit.
INFO: Build completed successfully, 2240 total actions

real 0m40.483s
user 0m0.048s
sys 0m0.058s
```

I think the `--jobs` option should only apply to `local` actions.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.