Custom JUnit TestRunner
- Dominant language
- Java
- Stars
- 7.2k
- Forks
- 820
- Avg merge
- 5h 9m
- Merged PRs (30d)
- 50
Description
The current pattern for tests leaves something to be desired:
- Each *(Positive|Negative)Cases.java file generally contains several checks. It would be nice if each check was an individual test case (`@Test`) so that it is easier to see exactly what is failing.
- Many *Test.java classes contain `String`ly typed source code. It's a pain in the neck to format it and add imports.
- This isn't Java 4 anymore, we have annotations! We could use an annotation instead of a comment to declare where we think errors are.
- Advantages:
- Easier to see what properties we can assert.
- Facilitates better documentation
- Disadvantages:
- We have to create a new class for the annotation
- It is harder to match an annotation type if you're only given a `String`. But we do have access to the AST...
My proposal:
- Create a `TestRunner` which will report each expected diagnostic as a separate test case with a specific name. Let's call it `ErrorProneRunner`.
- Annotate each *(Positive|Negative)Cases.java file with `@RunWith(ErrorProneRunner.class)`
- Create dummy packages in testdata for `String`ly typed source code and tell the `ErrorProneRunner` to do its thing (possibly with an annotation on *PositiveCases.java?)
- Replace `// BUG: Diagnostic contains: XXX` with `@ExpectDiagnostic("specific expectation message")`:
```
@interface ExpectDiagnostic {
String value();
String[] messageContains() default {};
String[] fixesContain() default {};
.
.
}
```
- We can also add the counterpoint `@DoNotExpectDiagnostic`
Thoughts?
Contributor guide
Assessment
This issue has not been assessed yet.