bazelbuild / bazelbuild/bazel-bench
Benchmarking incremental builds
- Dominant language
- Python
- Stars
- 42
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
**Description of the problem / feature request:**
Currently bazel-bench isn't able to benchmark incremental builds due to the following reasons:
1. Does not allow running `--warm_up=n` runs prior to the benchmark `--runs=n`
2. Does not allow toggling `--shutdown` and `--clean` to allow us to toggle whether or not to run `blaze shutdown / clean` (currently runs both by default).
3. Does not allow patching of code, should be done via specifying a `--patch_file`.
**Feature requests: what underlying problem are you trying to solve with this feature?**
Allows users to benchmark the incremental case, by:
1. Specifying `--warm_up` runs to warm up the blaze cache.
2. Do not `shutdown` or `clean` in between runs so that we run with a warm blaze cache.
3. Apply the patch file in-between runs to simulate code changes.
Contributor guide
Research direction
Start by locating the command-line handling for --warm_up, --shutdown, --clean, and --patch_file, then inspect how benchmark runs currently prepare and repeat builds. Done means users can warm the cache, control cleanup and shutdown between runs, and apply a patch file between benchmark runs to measure incremental builds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100