Bazel uses exactly 25% more resources (as defined via resource_set) than given with --local_cpu_resources
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 75
Description
### Description of the bug:
I have two actions that each declare CPU usage of 1000 by passing `resource_set` to `actions.run_shell`.
`bazel build --local_cpu_resources=1600 ...` runs both actions in parallel.
`bazel build --local_cpu_resources=1599 ...` runs them one after the other.
I would expect the cutoff to be between 1999 and 2000 instead.
### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
```
#!/bin/bash
set -euo pipefail
cat << 'EOF' > BUILD
load(":defs.bzl", "custom_bash_rule")
custom_bash_rule(
name = "file1",
command = "sleep 5; touch $OUTPUT",
output = "file1",
)
custom_bash_rule(
name = "file2",
command = "sleep 5; touch $OUTPUT",
output = "file2",
)
EOF
cat << 'EOF' > defs.bzl
# buildifier: disable=module-docstring
def _resource_set_callback(os_name, num_inputs):
return {"cpu": 1000}
def _custom_bash_rule_impl(ctx):
output_file = ctx.actions.declare_file(ctx.attr.output)
ctx.actions.run_shell(
command = "\n".join([
"set -euo pipefail",
'OUTPUT="' + output_file.path + '"',
ctx.attr.command,
]),
outputs = [output_file],
resource_set = _resource_set_callback,
)
return [DefaultInfo(files = depset(direct = [output_file]))]
custom_bash_rule = rule(
implementation = _custom_bash_rule_impl,
attrs = {
"command": attr.string(mandatory = True),
"output": attr.string(mandatory = True),
},
)
EOF
cat << 'EOF' > WORKSPACE
EOF
bazel clean --expunge
echo
echo "Building with --local_cpu_resources=1600"
bazel build --local_cpu_resources=1600 ...
bazel clean --expunge
echo
echo "Building with --local_cpu_resources=1599"
bazel build --local_cpu_resources=1599 ...
```
Output:
```
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
Building with --local_cpu_resources=1600
Starting local Bazel server and connecting to it...
INFO: Analyzed 2 targets (4 packages loaded, 7 targets configured).
INFO: Found 2 targets...
INFO: Elapsed time: 7.840s, Critical Path: 5.07s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
Building with --local_cpu_resources=1599
Starting local Bazel server and connecting to it...
INFO: Analyzed 2 targets (4 packages loaded, 7 targets configured).
INFO: Found 2 targets...
INFO: Elapsed time: 12.873s, Critical Path: 10.08s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
```
### Which operating system are you running Bazel on?
Ubuntu 20.04
### What is the output of `bazel info release`?
release 6.0.0
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?
_No response_
### Have you found anything relevant by searching the web?
_No response_
### Any other information, logs, or outputs that you want to share?
_No response_
Contributor guide
Research direction
Start by running the supplied BUILD and defs.bzl reproduction with --local_cpu_resources set to 1600 and 1599, focusing on actions.run_shell and resource_set scheduling. Trace the implementation of --local_cpu_resources and resource accounting, then add a regression test showing that two actions declaring 1000 CPU resources have the expected concurrency cutoff.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, shell
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100