bazelbuild / bazelbuild/bazel

Consistent resource management across actions and resource types

Open
#19,679 5 comments 2 reactions 0 assignees View on GitHub
P3 team-Local-Exec type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the feature request:

There are several way of defining and allocating resources for bazel:
- `--local_cpu_resources` and `--local_ram_resources` can be used to define the amount of cpu and ram available for bazel to consume. They both accept decimal values (float)
- `--local_extra_resources` can be used to model arbitrary abstract resources, also as floats
- The `cpu:` tag (or execution info) can be used to request more cpus for an action (defaults to 1)
- Only works for tests
- Only allows integer values, but it is modelled internally as a double
- The `resources::` can be used to request any amount of an arbitrary resource
- Only applied to tests
- Accepts float values
- As far as I understand, there is no way of requesting more memory (ram), even thou it is modelled internally as number of MB (as a double value)

I would like have fine-grained controll on all the resources types and all the actions, with an unified api, which stores all the values as doubles, something like:
- [x] `resources::` allows allocating any resource, including cpu and memory
- [x] `--local_resources==` can be used to specify the amount of any resource available for bazel to use. This would include `cpu` and `memory`
- [x] `--default_test_resources==,,,` allows setting the default allocation for test based on the test size.
- [ ] `--default_resources==` can be used to change the default allocation for all actions
- [ ] `resources::` can be applied to any action (not test only)

### Which category does this issue belong to?

Local Execution

### What underlying problem are you trying to solve with this feature?

In large, multi language repositories, the default values for resource allocation might not be suitable for all targets/actions. When that happens, bazel ends up over (or under) scheduling tasks, which affects performance and stability. With fine grained controll over the resource allocation, the build can be tweaked according to its needs.

### Which operating system are you running Bazel on?

N/A

### What is the output of `bazel info release`?

release 6.3.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing Bazel's existing resource flags, test-size defaults, and resource tags to understand how allocations differ between tests and other actions. The requested work is complete when default allocations can be configured for all actions and resource requests apply beyond tests, with coverage for CPU, memory, and arbitrary resources.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.