Using softAssert and "hard" Asserts together, softAsserts are swallowed by the "hard" Asserts

TestNG Version

7.1.0, latest from the maven repo https://mvnrepository.com/artifact/org.testng/testng

Issue

This issue is (probably) not one with testng per se but rather an architectural problem I have using testng and softAssertions. I found a solution and my questions are if there is a better solution to this and if there is a reason why I should not use my solution to this problem.

Essentially what I am doing is this:

softAssert.fail("SoftAssert fail");
assert.fail("assert fail");
softAssert.assertAll();

What happens is that assert.fail("assert fail"); obviously fails and the exception is shown as java.lang.AssertionError: assert fail but the softAssertion is never evaluated because of the Exception. The softAssertions will never be shown until the normal assert does not fail. But I want the failed softAsserts to be shown as well.

So what I ended up doing is this:
Simply extend SoftAssert so I can use the "doAssert" method, because it is protected in the original class.

public class MySoftAssert extends SoftAssert {
    @Override
    protected void doAssert(IAssert<?> a) {
        super.doAssert(a);
    }
}

Then define a HardAssert class, which also passes all asserts to 'the' softAssert object used by the tests to do assertions, and when the hardAssert fails it automatically evaluates all the softAsserts, which also includes the just done hardAssertion.

public class MyHardAssert extends SoftAssert {
    private final MySoftAssert softAssert;

    public MyHardAssert(MySoftAssert softAssert){
        this.softAssert = softAssert;
    }

    @Override
    public void onAssertFailure(IAssert<?> assertCommand, AssertionError ex) {
        super.onAssertFailure(assertCommand, ex);
        softAssert.assertAll("Hard Assert failed, also showing all soft asserts that failed before:");
    }

    @Override
    protected void doAssert(IAssert<?> a) {
        softAssert.doAssert(a);
        super.doAssert(a);
    }
}

Now I can do this:

softAssert = new MySoftAssert();
hardAssert = new MyHardAssert(softAssert);
softAssert.fail("softAssert fail");
hardAssert.fail("hardAssert fail");
...

and it will show both asserts as being failed.

java.lang.AssertionError: The following asserts failed:
	softAssert fail
	hardAssert fail

What I am also using is a test wrapper/listener that automatically evaluates all softAsserts at the end, but it is not called when the tests fail because of a 'normal' assertion. That wrapper and the hardAssert is inspired by code I found here: https://github.com/cbeust/testng/issues/1038#issuecomment-217466619

public class SoftAssertListener implements IHookable {

    @Override
    public void run(IHookCallBack iHookCallBack, ITestResult iTestResult) {
        iHookCallBack.runTestMethod(iTestResult);
        Class<?> testClass = iTestResult.getTestClass().getRealClass();
        Field field = null;
        SoftAssert softAssert = null;
        try {
            field = testClass.getField("softAssert");
            field.setAccessible(true);
            softAssert = (SoftAssert) field.get(iTestResult.getInstance());
        } catch (NoSuchFieldException | IllegalAccessException e) {
            e.printStackTrace();
        }

        if(softAssert != null) {
            softAssert.assertAll();
        }
    }
}

Is the issue reproductible on runner?

• IntelliJ

The issue provides a Java reproduction using SoftAssert, MySoftAssert, MyHardAssert, and SoftAssertListener but names no repository files or tests. Start by reproducing the behavior on TestNG 7.1.0 with a hard assertion before assertAll, then review the 14-comment discussion to determine the expected assertion lifecycle and whether a maintainable project change is actually agreed upon.