bazelbuild / bazelbuild/bazel-bench

Benchmarking incremental builds

Open
#153 2 comments 4 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
42
Forks
12
PR merge metrics
No merged PRs in 30d

Description

**Description of the problem / feature request:**

Currently bazel-bench isn't able to benchmark incremental builds due to the following reasons:

1. Does not allow running `--warm_up=n` runs prior to the benchmark `--runs=n`
2. Does not allow toggling `--shutdown` and `--clean` to allow us to toggle whether or not to run `blaze shutdown / clean` (currently runs both by default).
3. Does not allow patching of code, should be done via specifying a `--patch_file`.

**Feature requests: what underlying problem are you trying to solve with this feature?**

Allows users to benchmark the incremental case, by:

1. Specifying `--warm_up` runs to warm up the blaze cache.
2. Do not `shutdown` or `clean` in between runs so that we run with a warm blaze cache.
3. Apply the patch file in-between runs to simulate code changes.

Contributor guide

Open the contributing guide

Research direction

Start by locating the command-line handling for --warm_up, --shutdown, --clean, and --patch_file, then inspect how benchmark runs currently prepare and repeat builds. Done means users can warm the cache, control cleanup and shutdown between runs, and apply a patch file between benchmark runs to measure incremental builds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.