android / android/android-test

Limitation of ActivityScenario's Launch Method

Open
#143 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.2k
Forks
342
Avg merge
11h 29m
Merged PRs (30d)
2

Description

### Description
As referenced in [my original AndroidX Discuss Post](https://groups.google.com/forum/#!topic/androidx-test-discuss/9UYY-2HZJ38) and [my Stack Overflow post](https://stackoverflow.com/questions/53598321/androidx-test-activityscenario-java-lang-assertionerror-activity-never-becomes/53638398#53638398) and [this new stack overflow post about FragmentScenario](https://stackoverflow.com/questions/53629258/instrumental-testing-with-fragmentscenario/53638246#53638246), there seems to be a limitation of the ActivityScenario API when using `ActivityScenario.launch()` method because it only waits for two possible `Lifecycle.State`s `RESUMED` and `DESTROYED`.

**Claim:**
There is a limitation within the `launch(Intent startActivityIntent)` method of the ActivityScenario API. It waits for the Activity to be `Lifecycle.STATE.RESUMED` or `DESTROYED` and if it isn't within 4.5 seconds then it throws this error.

**Context:**
My application uses an `IndexActivity` to load a config which instructs the application on certain API calls to make. However, immediately after it loads a `DialogActivity` and the `IndexActivity` goes into `STOPPED`. On accepting terms within the `DialogActivity` the `IndexActivity` goes back into `RESUMED` and then ActivityScenario works properly. With my tests, there was a race condition on whether Espresso could click through the terms within 4.5 seconds to get the `IndexActivity` to be `RESUMED` or whether this error would throw before that. It would take major refactoring to enable another Activity to be launched with ActivityScenario so that was not an option.

**The Fix**
Within `public static ActivityScenario launch(Intent startActivityIntent)` of Activity Scenario, check the logic `scenario.waitForActivityToBecomeAnyOf(State.RESUMED, State.DESTROYED);`

If you can create your own custom Activity Scenario and adjust this line of code to be something like `scenario.waitForActivityToBecomeAnyOf(State.STOPPED, State.DESTROYED);` then it will theoretically work for you. You can then use ActivityScenario again to move the Activity into whatever Lifecycle State you want.

**OR** just use the old https://developer.android.com/reference/androidx/test/rule/ActivityTestRule

**TL;DR**
This is happening because the Lifecycle.State of your Activity is not either of the two specific lifecycle states `ActivityScenario.Launch()` waits for, `RESUMED` or `DESTROYED`. Your activity is probably in the background of a dialog or another edge-case situation that was not thought about when creating the API.

Full StackTrace for Test here:

```
10:54:42 V/InstrumentationResultParser: java.lang.AssertionError: Activity never becomes requested state "[RESUMED]" (last lifecycle transition = "STOPPED")
10:54:42 V/InstrumentationResultParser: at androidx.test.core.app.ActivityScenario.waitForActivityToBecomeAnyOf(ActivityScenario.java:228)
10:54:42 V/InstrumentationResultParser: at androidx.test.core.app.ActivityScenario.moveToState(ActivityScenario.java:368)
10:54:42 V/InstrumentationResultParser: at com.myapplication.android.test.HomeTest.launchActivity(HomeTest.java:30)
10:54:42 V/InstrumentationResultParser: at java.lang.reflect.Method.invoke(Native Method)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
10:54:42 V/InstrumentationResultParser: at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
10:54:42 V/InstrumentationResultParser: at androidx.test.internal.runner.junit4.statement.RunBefores.evaluate(RunBefores.java:76)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
10:54:42 V/InstrumentationResultParser: at androidx.test.ext.junit.runners.AndroidJUnit4.run(AndroidJUnit4.java:104)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.Suite.runChild(Suite.java:128)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.Suite.runChild(Suite.java:27)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
10:54:42 V/InstrumentationResultParser: at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
10:54:42 V/InstrumentationResultParser: at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
10:54:42 V/InstrumentationResultParser: at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
10:54:42 V/InstrumentationResultParser: at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:388)
10:54:42 V/InstrumentationResultParser: at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2075)
```
### Steps to Reproduce
1) Start Activity with ActivityScenario
2) Activity has some condition where either terms have to be accepted or waits a few seconds before going into `RESUMED` state, due to pop-up modal or dialog

### Expected Results
**It seems this is not a bug but is rather a restriction of the API**
But, the expected behavior would be to either allow `STOPPED` to be waited for within the launch() method, or to allow the user to specify which states they want to wait for.

### Actual Results
ActivityScenario wants your activity to be either `RESUMED` or `DESTROYED`, and if it isn't after 45000 milliseconds then it throws the error above.

### AndroidX Test and Android OS Versions
```
android {
defaultConfig {
// Specifies instrumentation which connects the test package and the application package
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"

// The package name of the test app
testApplicationId 'com.myapplication.android.test'

// The following argument makes the Android Test Orchestrator run its
// "pm clear" command after each test invocation. This command ensures
// that the app's state is completely cleared between tests.
testInstrumentationRunnerArguments clearPackageData: 'true'
}

testOptions {
animationsDisabled = true
execution 'ANDROIDX_TEST_ORCHESTRATOR'
}

useLibrary 'android.test.runner'
useLibrary 'android.test.base'
useLibrary 'android.test.mock'
}

dependencies {
// Core library
androidTestImplementation 'androidx.test:core:1.0.0'

// AndroidJUnitRunner and JUnit Rules
androidTestImplementation 'androidx.test:runner:1.1.0'
androidTestImplementation 'androidx.test:rules:1.1.0'
androidTestUtil 'androidx.test:orchestrator:1.1.0'

// Assertions
androidTestImplementation 'androidx.test.ext:junit:1.0.0'
androidTestImplementation 'androidx.test.ext:truth:1.0.0'
androidTestImplementation 'com.google.truth:truth:0.42'

// Espresso dependencies
androidTestImplementation 'androidx.test.espresso:espresso-core:3.1.0'
androidTestImplementation 'androidx.test.espresso:espresso-contrib:3.1.0'
androidTestImplementation 'androidx.test.espresso:espresso-intents:3.1.0'
androidTestImplementation 'androidx.test.espresso:espresso-accessibility:3.1.0'
androidTestImplementation 'androidx.test.espresso:espresso-web:3.1.0'
androidTestImplementation 'androidx.test.espresso.idling:idling-concurrent:3.1.0'

// The following Espresso dependency can be either "implementation"
// or "androidTestImplementation", depending on whether you want the
// dependency to appear on your APK's compile classpath or the test APK
// classpath.
androidTestImplementation 'androidx.test.espresso:espresso-idling-resource:3.1.0'

}
```
### Link to a public git repo demonstrating the problem:
None right now.

Contributor guide

Open the contributing guide

Research direction

Start in ActivityScenario.java at launch(Intent startActivityIntent), moveToState, and waitForActivityToBecomeAnyOf, using the reported STOPPED-versus-RESUMED lifecycle scenario as the reproduction. Determine whether launch should support STOPPED or caller-selected states, then add coverage showing the intended state is accepted without the timeout failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, java
Domain
mobile, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.