eclipse-jdt / eclipse-jdt/eclipse.jdt.ui
Rerun Tests - Failures first behaves inconsistently
- Dominant language
- Java
- Stars
- 59
- Forks
- 127
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 37
Description
Given two JUnit 6 test classes with three tests each, six in total. Of these three tests one is failing. I would expect the "Rerun Tests - Failures first" to sort the test classes and methods such that classes with failing tests are executed first and within the classes, failing methods go first. In practice however the results are inconsistent.
For example this works as expected:
```java
package com.example.project;
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;
class TestA {
@Test
void test1() {
System.out.println("A1");
}
@Test
void test2() {
System.out.println("A2");
assertEquals(1, 2);
}
@Test
void test3() {
System.out.println("A3");
}
}
```
This will print out
```
A1
A2
A3
```
And clicking to rerun failures first outputs as expected:
```
A2
A1
A3
```
But then adding a second class for example:
```java
package com.example.project;
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
class TestB {
@Test
void test1() {
System.out.println("B1");
}
@Test
void test2() {
System.out.println("B2");
}
@Test
void test3() {
System.out.println("B3");
}
}
```
And running the test in the `com.example.project` normally outputs:
```
B1
B2
B3
A1
A2
A3
```
And clicking to rerun failures first unexpectedly does not change the order in which the tests were ran:
```
B1
B2
B3
A1
A2
A3
```
Versions:
JUnit: 6.0.1
Eclipse: 2025-12 (4.38.0)
Contributor guide
Research direction
Reproduce the issue in Eclipse 2025-12 with JUnit 6.0.1 using the two Java classes shown, first with one failing method and then with both classes. Compare the normal run with “Rerun Tests - Failures first”; done means the class containing failures runs first and its failing methods precede passing methods consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100