android / android/android-test

Exceptions in after methods hide exceptions in test methods

Open
#670 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.2k
Forks
342
Avg merge
11h 29m
Merged PRs (30d)
2

Description

### Description
Exceptions from `@After` methods hide exceptions from test methods, which sometimes makes debugging tests difficult. Also this behavior differs from desktop JUnit 4 behavior.

### Steps to Reproduce
1. Setup a test class where both a test method and an after method throw.
2. Run the test class
3. Check the results (Gradle console output, HTML and XML reports, run log in Android Studio)

### Expected Results (based on desktop JUnit behavior)
Exceptions from both the test method and the after method are reported:
- Exceptions from both methods should be reported in Gradle console output
- HTML report should include exceptions from both methods
- XML report should have 2 failure nodes for the test
- Execution log in Android Studio should contain exception messages from both methods

### Actual Results
Only the exception thrown from the after method is reported by Gradle, HTML and XML reports, Android Studio.

### AndroidX Test and Android OS Versions
Tested with API levels 24, 26, 30 and AndroidX Test 1.2.0.

### Link to a public git repo demonstrating the problem:
https://github.com/TarCV/android-junit-issue
Run instrumented tests to demonstrate the issue.
I've also added similar normal unit tests to make it easier comparing AndroidX test results with desktop test results.

Contributor guide

Open the contributing guide

Research direction

Start by running the instrumented tests from the linked public reproduction repository and compare AndroidX Test results with the normal unit tests. Inspect how failures are represented in Gradle console output, HTML and XML reports, and the Android Studio execution log. Done means both test-method and @After exceptions appear in each requested result, including two XML failure nodes.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, java
Domain
mobile, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.