apache / apache/maven-surefire

[SUREFIRE-1480] parallel tests may produce invalid .xml report

Open
#2,000 22 comments 0 reactions 0 assignees View on GitHub
bug priority:major
Dominant language
Java
Stars
461
Forks
588
Avg merge
1d 8h
Merged PRs (30d)
19

Description

**[Mark Lehky](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mlehky)** opened **[SUREFIRE-1480](https://issues.apache.org/jira/browse/SUREFIRE-1480?redirect=false)** and commented

Relevant software:
* Jenkins 2.108
* Maven 3.??
* JUnit 4.12
* maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well)

I have a testsuite (one JUnit class) that contains multiple tests (multiple JUnit methods), which are all run in parallel. Some of the tests may be ignore using JUnit `Assume`.

On occasion (not 100% reproducible), the resulting report will contain an entry like:

```
< message="Skip test!">
```

The correct entry, as is produced most of the time, should be:

```

```

The invalid formatted XML, when run in Jenkins, results in the test being flagged as failed, and Jenkins simply has the message: "TEST-....xml.[failed-to-read]" (the dots are replaced with the correct filename!).

---

**Affects:** 2.20.1

**Attachments:**
- [FailedXMLReport.txt](https://issues.apache.org/jira/secure/attachment/12912747/FailedXMLReport.txt) (_3.65 kB_)
- [Stacktrace_failedTest.txt](https://issues.apache.org/jira/secure/attachment/12912748/Stacktrace_failedTest.txt) (_10.96 kB_)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the attached FailedXMLReport.txt and Stacktrace_failedTest.txt, then reproduce the issue with parallel JUnit 4.12 tests using Assume and maven-failsafe-plugin 2.20.1. Trace how the test report is written when tests are skipped concurrently. Done means the report consistently contains valid XML with a self-closing skipped element and Jenkins can read it.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.