bazelbuild / bazelbuild/bazel

Improve performance of undeclared output discovery

Open
#29,464 0 comments 0 reactions 1 assignee Claimed by @john-flanagan View on GitHub
P1 team-Performance type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

Generating the undeclared outputs manifest can become noticeably slow when tests generate a large number of undeclared outputs.

XCResult files from `xcodebuild` contain hundreds, or even thousands, of files within them for large test bundles and can take over 10 seconds to traverse with the current implantation.

Our project has a unit test bundle that takes roughly 12 seconds for the tests to complete, but then another 10-12 seconds is consumed by generating the undeclared output manifest for the XCResult file it creates.

### Which category does this issue belong to?

Performance

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Create a `sh_test` target that writes 2,000 files.

```bash
#!/usr/bin/env bash

set -euo pipefail

for i in $(seq 1 2000); do
printf 'content %d\n' "$i" > "${TEST_UNDECLARED_OUTPUTS_DIR}/output_${i}.txt"
done

echo "Writing files finished"
```

Run the test with `--test_output=streamed` and observe the delay between "Writing files finished" being printed to the console and the test finishing.

### Which operating system are you running Bazel on?

macOS 26.4.1

### What is the output of `bazel info release`?

release 9.1.0

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

```text
https://github.com/bazelbuild/bazel.git
42b1f15e1063cffdf9711394bbf8020c0d72e24a
```

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.