bazelbuild / bazelbuild/bazel

Using --local_extra_resources limits concurrency

Open
#18,153 12 comments 0 reactions 1 assignee Assigned to @zhengwei143 View on GitHub
P2 team-Local-Exec type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

If some tests require extra resources (via `--local_extra_resources`) but others don't, the concurrency of tests that do not require the extra resource is limited by tests that do require the extra resources being scheduled but not starting. These tests that are scheduled but not started count as a concurrent running job, but sit there doing nothing when a job that does not require the resource could be running.

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

A reproducer is available at the following repository: https://github.com/cameron-martin/bazel-extra-resources-scheduling-bug

Tests can be run like so:

```
bazel test //:all
```

Half of these tests do not require extra resources, so concurrency should not be limited until these complete. Instead, the number of concurrent jobs drops to way below the maximum since tests that require an unavailable resource are scheduled but cannot yet start.

### Which operating system are you running Bazel on?

Ubuntu 22.04

### What is the output of `bazel info release`?

release 6.1.2

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

https://bazelbuild.slack.com/archives/CA31HN1T3/p1681922669112529

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.