openrewrite / openrewrite/rewrite-testing-frameworks

`ExpectedExceptionToAssertThrows` emits uncompilable code when `expect(Matcher)` is given a matcher narrower than `Throwable`

Open
#1,124 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
100
Forks
105
Avg merge
2h 25m
Merged PRs (30d)
9

Description

What happens

ExpectedExceptionToAssertThrows rewrites

thrown.expect(someMatcher);
doSomething();

into

Throwable exception = assertThrows(Exception.class, () -> doSomething());
assertThat(exception, someMatcher);

The captured variable is always declared Throwable. MatcherAssert.assertThat is declared

public static <T> void assertThat(T actual, Matcher<? super T> matcher);

so the emitted call requires Matcher<? super Throwable>. If the matcher's type argument is a
subtype of Throwable — as it is for any custom matcher written against a specific exception
class — no T can satisfy both constraints (Throwable <: T and T <: TheException), and the
migrated file does not compile.

Minimal reproduction

AppError.java

package com.example;

public class AppError extends RuntimeException {
    private final int code;

    public AppError(int code) {
        super("app error " + code);
        this.code = code;
    }

    public int getCode() {
        return code;
    }
}

TypedMatcherTest.java

package com.example;

import org.hamcrest.CustomMatcher;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;

public class TypedMatcherTest {

    @Rule public ExpectedException thrown = ExpectedException.none();

    static class AppErrorMatcher extends CustomMatcher<AppError> {
        private final int expectedCode;

        AppErrorMatcher(int expectedCode) {
            super("AppError with code " + expectedCode);
            this.expectedCode = expectedCode;
        }

        @Override
        public boolean matches(Object object) {
            return (object instanceof AppError) && ((AppError) object).getCode() == expectedCode;
        }
    }

    @Test
    public void failsWithCode42() {
        thrown.expect(new AppErrorMatcher(42));
        throw new AppError(42);
    }
}

Run org.openrewrite.java.testing.junit5.JUnit4to5Migration. Result:

    @Test
    public void failsWithCode42() {
        Throwable exception = assertThrows(Exception.class, () -> {
            throw new AppError(42);
        });
        assertThat(exception, new AppErrorMatcher(42));
    }
[ERROR] TypedMatcherTest.java:[30,9] no suitable method found for
        assertThat(java.lang.Throwable,com.example.TypedMatcherTest.AppErrorMatcher)

Note the original test was correct and passing: CustomMatcher.matches takes Object, so the
narrow type argument never mattered at runtime. ExpectedException.expect accepts Matcher<?> and
performs the check reflectively, so the narrowing was invisible under JUnit 4. The migration is what
makes the static type load-bearing.

Why this matters

Whether the migrated code compiles depends entirely on how the matcher class happens to be
declared, not on anything visible at the call site — and because JUnit4to5Migration removes the
junit:junit dependency in the same run, there is no falling back to the JUnit 4 form either:

Matcher declaration Type argument Compiles after migration
extends CustomMatcher<CommandError> narrower no
extends TypeSafeMatcher<Exception> narrower no
extends TypeSafeMatcher<Throwable> exact yes
extends BaseMatcher<Throwable> exact yes
isA(X.class) / instanceOf(X.class) free type variable yes

This is unavoidable Java generics variance once a matcher's type is fixed at its declaration, not an
OpenRewrite-specific quirk: MatcherAssert.assertThat(Throwable, Matcher<? super Throwable>)
genuinely cannot accept a Matcher<AppError> argument — compiling the narrowed case directly against
hamcrest with javac fails with inference variable T has incompatible bounds: upper bounds: AppError,Object; lower bounds: Throwable.

Why the existing tests don't cover it

The recipe already handles two neighbouring cases correctly, and they are the ones the fixtures
exercise:

// (a) class literal — becomes assertThrows(AppError.class, ...) with no assertThat at all
thrown.expect(AppError.class);

// (b) isA / instanceOf — keeps the Throwable variable, and still compiles
thrown.expect(isA(AppError.class));

Case (b) compiles only by accident of Hamcrest's declaration:

public static <T> Matcher<T> isA(java.lang.Class<?> type);
public static <T> Matcher<T> instanceOf(java.lang.Class<?> type);

T is a free type variable — the class argument is Class<?>, not Class<T> — so T infers to
Throwable at the call site and Matcher<? super Throwable> is satisfied. Every
org.hamcrest.Matchers factory used in the fixtures behaves this way.

A hand-written matcher does not. class AppErrorMatcher extends CustomMatcher<AppError> fixes the
type argument at its declaration, so nothing can widen it at the call site. The recipe's
Throwable exception = … template is therefore safe only for matchers whose type parameter is free
or exactly Throwable. Both control cases above are real, passing tests in
ExpectedExceptionToAssertThrowsTest; there is no fixture anywhere using a hand-written
CustomMatcher<SpecificType> or TypeSafeMatcher<SpecificType>.

In ExpectedExceptionToAssertThrows.visitBlock, the declaration is hardcoded:

String exceptionDeclParam = getCursor().pollMessage(HAS_MATCHER) != null ? "Throwable exception = " : "";
Object exceptionClass = getCursor().pollMessage(EXCEPTION_CLASS);
if (exceptionClass == null) {
    exceptionClass = "Exception.class";
}

EXCEPTION_CLASS is only ever populated from expect(java.lang.Class)
(EXPECTED_EXCEPTION_CLASS_MATCHER), so a expect(Matcher)-only test always gets Throwable /
Exception.class. The identifier is then built with a fixed type:

J.Identifier exceptionIdentifier = new J.Identifier(Tree.randomId(),
        Space.EMPTY, Markers.EMPTY, emptyList(), "exception",
        JavaType.ShallowClass.build("java.lang.Throwable"), null);

and the assertion template is

template = "assertThat(#{any(java.lang.Throwable)}, #{any(org.hamcrest.Matcher)})";

The recipe already has machinery for narrowing the exception type — the
extractClassLiteralFromInstanceOfMatcher / maybeAssertInstanceOfForCause path pulls a class
literal out of an isA(X.class) argument — but it is scoped to expectCause(matcher) only; it never
runs for the plain expect(matcher) path this bug lives in, so there's no partial handling on this
path to build on, only a model for what the fix could reuse.

Resolving the matcher's type argument and using it for both the assertThrows class and the local
variable would fix the reproduction above:

AppError exception = assertThrows(AppError.class, () -> {
    throw new AppError(42);
});
assertThat(exception, new AppErrorMatcher(42));

The matcher expression's JavaType gives the argument directly: walk the JavaType.Parameterized
supertype chain to org.hamcrest.Matcher<T> and take T, applying this only when T is a
Throwable subtype and falling back to today's behaviour otherwise.

One semantic caveat worth deciding deliberately: assertThrows(AppError.class, …) fails when a
different exception type is thrown, whereas ExpectedException with a matcher would run the
matcher and report a mismatch. Both are failures, and the assertThrows message is arguably the
better one, but it is a change in which assertion reports the problem. If narrowing is unwanted, the
alternative is to leave the construct alone — skip the method and retain the JUnit 4 dependency — so
the situation surfaces as an unmigrated test rather than a broken build.

Environment
  • rewrite-testing-frameworks 3.44.0 (same code in 3.46.0-SNAPSHOT)
  • rewrite-maven-plugin 6.41.0
  • Recipe: org.openrewrite.java.testing.junit5.JUnit4to5Migration

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in ExpectedExceptionToAssertThrows.visitBlock, especially the hardcoded exception declaration, exception identifier, and assertThat template. Add a regression fixture to ExpectedExceptionToAssertThrowsTest using a typed CustomMatcher, then run the recipe tests. Done means the typed matcher migration compiles while the existing class-literal and isA/instanceOf cases still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing-qa, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.