bazelbuild / bazelbuild/bazel-bench

Missing benchmark

Open
#40 2 comments 0 reactions 0 assignees View on GitHub
bug P2
Dominant language
Python
Stars
42
Forks
12
PR merge metrics
No merged PRs in 30d

Description

**Description of the problem / feature request:**

bazel-bench did not run on all specified bazel commits.

**Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.**

Ran the following command:
```
bazel run :benchmark -- --bazel_commits ffad58728d850b465a070d25b037ea1bced0f91f,4782b8872c3bef8df56f3714ffd005dd3dcc95f7 --bazel_source /usr/local/google/home/twerth/bazel-up --collect_memory --data_directory /tmp/bazel-g1_2 --project_commits ffad58728d850b465a070d25b037ea1bced0f91f --project_source /usr/local/google/home/twerth/bazel-up --runs 20 --collect_json_profile -- build --compilation_mode=opt //src:bazel
```

Note that I am using the same local repository as bazel and project source - is that causing any problems?

From the logs:
```
INFO: Running command line: bazel-out/k8-py2-fastbuild/bin/benchmark --bazel_commits ffad58728d850b465a070d25b037ea1bced0f91f,4782b8872c3bef8df56f3714ffd005dd3dcc95f7 --bazel_source /usr/local/google/home/twerth/bazel-up --collect_memory --data_directory /tmp/bazel-g1_2 --project_commits ffad58728d850b465a070d25b037ea1bced0f91f --project_source /usr/local/google/home/twerth/bazel-up --runs 20 --collect_json_profile -- build '--compilation_mode=opt' //src:bazel
...
I0619 07:56:51.347815 139708403218240 logger.py:33] Preparing bazelbuild/bazel repository.
I0619 07:56:51.347987 139708403218240 logger.py:33] Path /tmp/.bazel-bench/bazel exists. Updating...
I0619 07:56:51.811319 139708403218240 logger.py:33] Preparing /usr/local/google/home/twerth/bazel-up clone.
I0619 07:56:51.811598 139708403218240 logger.py:33] Path /tmp/.bazel-bench/project-clones/cbf1f465934e5495d5024f0a2bd99ea5 exists. Updating...
I0619 07:56:52.261014 139708403218240 logger.py:33] Binary exists at /tmp/.bazel-bench/bazel-bin/ffad58728d850b465a070d25b037ea1bced0f91f/bazel, reusing...
I0619 07:56:52.328670 139708403218240 logger.py:33] Pre-fetching external dependencies & exporting build event json to /tmp/.bazel-bench/out/build_env.json...
```

So it is already missing in the preparation phase.

Contributor guide

Open the contributing guide

Research direction

Start by running the reported `:benchmark` command and trace the preparation phase around `logger.py:33`. Compare the requested Bazel commits with the benchmark data produced after preparation, including the case where `--bazel_source` and `--project_source` use the same repository. Done means every specified commit is prepared and benchmarked.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.