bazelbuild / bazelbuild/bazel

bazel test: Generate JUnit XML reports for FAILED TO BUILD test targets

Open
#24,447 0 comments 0 reactions 0 assignees View on GitHub
P2 team-Rules-Java type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the feature request:

Currently, when running `bazel test`, JUnit XML reports are only generated for tests that compiled successfully but failed during execution. However, no XML reports are generated for targets that fail during the build phase (`FAILED TO BUILD`).

This makes it challenging to track and analyze build failures in CI systems that rely on JUnit XML reports for test result aggregation and visualization,

## Current Behavior
When a test target fails to build:

```bash
FAILED: Build did NOT complete successfully
//xla/service/gpu/runtime:dynamic_slice_thunk_test_gpu_v100 FAILED TO BUILD
```
No XML report is generated.

## Proposed Behavior
- Generate XML reports for `FAILED TO BUILD` targets.
- Include build error messages in the XML report.
- Treat build failures in the testing phase as test failures with appropriate error classification.

```xml




[Build error details]


```

### Which category does this issue belong to?

CLI, Core

### What underlying problem are you trying to solve with this feature?

CI systems miss test build failures in their test reports because no XML is generated for `FAILED TO BUILD` cases.
For now, we've had to implement custom scripts to generate XML reports for build failures.

### Which operating system are you running Bazel on?

Ubuntu 22.04.3

### What is the output of `bazel info release`?

release 6.5.0

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

```text
Internal; cannot provide
```

### Have you found anything relevant by searching the web?

N/A

### Any other information, logs, or outputs that you want to share?

Our current workaround:

```
detailed_error=$(echo "$bazel_output" | awk '
/error:/ { p=1 }
/FAILED: Build did NOT complete successfully/ { exit }
p { print }
' || true)

# Create an XML report for the build failure
cat < "$report_dir/$xml_filename"




Build target $failed_target failed to build.

Build error:
$(echo "$detailed_error" | sed 's/&/\&/g; s//\>/g; s/"/\"/g; s/'"'"'/\'/g')


EOF
```

Contributor guide

Open the contributing guide

Research direction

Start by tracing Bazel's existing JUnit XML generation for execution failures and the handling of `FAILED TO BUILD` targets during `bazel test`. Compare the current reports with the proposed XML and verify that build diagnostics are represented as errors for each failed target.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, cli, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.