bazelbuild / bazelbuild/bazel

Enhance "actuallyCompleteAction" algorithm for better build performance

Open
#18,614 5 comments 0 reactions 0 assignees View on GitHub
P2 team-Remote-Exec type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

`actuallyCompleteAction` adversely affects our build performance and has been a constant bottleneck.

As we analyze the step named 'Bundling, processing and signing App`, it is primarily composed of 2 parts.
1. `BundleTreeApp`: 272ms
2. `actuallyCompleteAction`: 12s 825ms

We can see that almost 100% of the time of this step is the overhead introduced by `actuallyCompleteAction`.
![Screenshot 2023-06-07 at 6 51 37 PM](https://github.com/bazelbuild/bazel/assets/11925399/c990d6de-f568-43dc-80bd-086017d3ce86)

This happens on every build and comes with a significantly high wait time cost as artifacts are huge. Any improvements here would be super helpful.

Related threads: https://github.com/bazelbuild/bazel/pull/18194, https://github.com/bazelbuild/bazel/issues/17009, https://github.com/bazelbuild/bazel/issues/10702

Below is a repro that demonstrates the bottleneck.

Repro on macOS:
```
bazel build //:App
```
[test_bundling2.zip](https://github.com/bazelbuild/bazel/files/11683272/test_bundling2.zip)

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Repro on macOS:
```
bazel build //:App
```
[test_bundling2.zip](https://github.com/bazelbuild/bazel/files/11683272/test_bundling2.zip)

### Which operating system are you running Bazel on?

macOS

### What is the output of `bazel info release`?

release 6.2.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

_No response_

### Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the attached test_bundling2.zip and reproduce the issue using `bazel build //:App` on macOS. Profile the “Bundling, processing and signing App” step, focusing on `BundleTreeApp` and `actuallyCompleteAction`; done means reducing the reported `actuallyCompleteAction` overhead in this reproduction.

Written by the indexing model from the issue text.

Assessment

Domain
build-system, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.