bazel test: Generate JUnit XML reports for FAILED TO BUILD test targets
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the feature request:
Currently, when running `bazel test`, JUnit XML reports are only generated for tests that compiled successfully but failed during execution. However, no XML reports are generated for targets that fail during the build phase (`FAILED TO BUILD`).
This makes it challenging to track and analyze build failures in CI systems that rely on JUnit XML reports for test result aggregation and visualization,
## Current Behavior
When a test target fails to build:
```bash
FAILED: Build did NOT complete successfully
//xla/service/gpu/runtime:dynamic_slice_thunk_test_gpu_v100 FAILED TO BUILD
```
No XML report is generated.
## Proposed Behavior
- Generate XML reports for `FAILED TO BUILD` targets.
- Include build error messages in the XML report.
- Treat build failures in the testing phase as test failures with appropriate error classification.
```xml
[Build error details]
```
### Which category does this issue belong to?
CLI, Core
### What underlying problem are you trying to solve with this feature?
CI systems miss test build failures in their test reports because no XML is generated for `FAILED TO BUILD` cases.
For now, we've had to implement custom scripts to generate XML reports for build failures.
### Which operating system are you running Bazel on?
Ubuntu 22.04.3
### What is the output of `bazel info release`?
release 6.5.0
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse HEAD` ?
```text
Internal; cannot provide
```
### Have you found anything relevant by searching the web?
N/A
### Any other information, logs, or outputs that you want to share?
Our current workaround:
```
detailed_error=$(echo "$bazel_output" | awk '
/error:/ { p=1 }
/FAILED: Build did NOT complete successfully/ { exit }
p { print }
' || true)
# Create an XML report for the build failure
cat < "$report_dir/$xml_filename"
Build target $failed_target failed to build.
Build error:
$(echo "$detailed_error" | sed 's/&/\&/g; s//\>/g; s/"/\"/g; s/'"'"'/\'/g')
EOF
```
Contributor guide
Research direction
Start by tracing Bazel's existing JUnit XML generation for execution failures and the handling of `FAILED TO BUILD` targets during `bazel test`. Compare the current reports with the proposed XML and verify that build diagnostics are represented as errors for each failed target.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, cli, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100