eclipse-jdt / eclipse-jdt/eclipse.jdt.ui

Rerun Tests - Failures first behaves inconsistently

Open
#2,710 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
59
Forks
127
Avg merge
1d 8h
Merged PRs (30d)
37

Description

Given two JUnit 6 test classes with three tests each, six in total. Of these three tests one is failing. I would expect the "Rerun Tests - Failures first" to sort the test classes and methods such that classes with failing tests are executed first and within the classes, failing methods go first. In practice however the results are inconsistent.

For example this works as expected:

```java
package com.example.project;

import static org.junit.jupiter.api.Assertions.assertEquals;

import org.junit.jupiter.api.Test;

class TestA {

@Test
void test1() {
System.out.println("A1");
}

@Test
void test2() {
System.out.println("A2");
assertEquals(1, 2);

}

@Test
void test3() {
System.out.println("A3");
}

}
```

This will print out

```
A1
A2
A3
```

And clicking to rerun failures first outputs as expected:

```
A2
A1
A3
```

But then adding a second class for example:

```java
package com.example.project;

import static org.junit.jupiter.api.Assertions.assertEquals;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;

class TestB {

@Test
void test1() {
System.out.println("B1");
}

@Test
void test2() {
System.out.println("B2");
}

@Test
void test3() {
System.out.println("B3");
}

}
```

And running the test in the `com.example.project` normally outputs:

```
B1
B2
B3
A1
A2
A3
```

And clicking to rerun failures first unexpectedly does not change the order in which the tests were ran:

```
B1
B2
B3
A1
A2
A3
```

Versions:

JUnit: 6.0.1
Eclipse: 2025-12 (4.38.0)

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue in Eclipse 2025-12 with JUnit 6.0.1 using the two Java classes shown, first with one failing method and then with both classes. Compare the normal run with “Rerun Tests - Failures first”; done means the class containing failures runs first and its failing methods precede passing methods consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.