jenkinsci / jenkinsci/junit-plugin
[JENKINS-31187] Not all failure cases reported in JUnit test result report
- Dominant language
- Java
- Stars
- 87
- Forks
- 351
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 2
Description
Hi,
JUnit allows to collect multiple test failures, eg. using ErrorCollector rule. But JUnit Plugin reports only first failure.
For example test result may looks like this:
...
"com.SomeTestClass" name="someTestName" time="7.718">
"Some error 1 message" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: Some error 1 message
at Some failure 1 stack trace...
"Some error 2 message" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: Some error 2 message
at Some failure 2 stack trace...
...
From above snippet only "Some error 1 message" will be reported, junitResults.xml:
...
<case>
7.718
com.SomeTestClass
someTestName
false
junit.framework.AssertionFailedError: Some error 1 message
at Some failure 1 stack trace...
Some error 1 message
0
case>
..
Is it possible for JUnit Plugin to report all failures?
---
Originally reported by abeben, imported from: Not all failure cases reported in JUnit test result report
Raw content of original issue
Hi,
JUnit allows to collect multiple test failures, eg. using ErrorCollector rule. But JUnit Plugin reports only first failure.For example test result may looks like this:
...
<testcase classname="com.SomeTestClass" name="someTestName" time="7.718">
<failure message="Some error 1 message" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: Some error 1 message
at Some failure 1 stack trace...
</failure>
<failure message="Some error 2 message" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: Some error 2 message
at Some failure 2 stack trace...
</failure>
</testcase>
...From above snippet only "Some error 1 message" will be reported, junitResults.xml:
...
<case>
<duration>7.718</duration>
<className>com.SomeTestClass</className>
<testName>someTestName</testName>
<skipped>false</skipped>
<errorStackTrace>junit.framework.AssertionFailedError: Some error 1 message
at Some failure 1 stack trace...
</errorStackTrace>
<errorDetails>Some error 1 message</errorDetails>
<failedSince>0</failedSince>
</case>
..Is it possible for JUnit Plugin to report all failures?
environment
```
Jenkins ver. 1.619
JUnit Plugin ver. 1.9
```
Contributor guide
Research direction
The issue does not name a source file, test, or entry point. Start by locating the JUnit plugin code that parses multiple failure elements and produces the Jenkins test result, then add coverage for the two-failure XML example and confirm both failures are reported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100