android / android/nowinandroid
[Bug]: ScrollForYouFeedBenchmark tests are failing with Macrobenchmark library v1.2.0-alpha15
- Dominant language
- Kotlin
- Stars
- 21.8k
- Forks
- 4.6k
- Avg merge
- 19h 20m
- Merged PRs (30d)
- 2
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Is there a StackOverflow question about this issue?
- [X] I have searched StackOverflow
### What happened?
`ScrollForYouFeedBenchmark` tests are failing with the Macrobenchmark library v1.2.0-alpha15. It looks like an undocumented behavior change has been [released](https://developer.android.com/jetpack/androidx/releases/benchmark#version_12_2) on how `setupBlock` and `measureBlock` runs with `startupMode` set to `StartupMode.COLD`. With the current stable [version v1.1.1](https://developer.android.com/jetpack/androidx/releases/benchmark#version_111_3) , `measureBlock` runs after the Activity is started implicitly by the Android system. In the newer version v1.2.0-xxx , we must start the Activity before running the `measureBlock` block
### With Macrobenchmark v1.1.1:
```kotlin
private fun scrollFeed(compilationMode: CompilationMode) = benchmarkRule.measureRepeated(
// ...
startupMode = StartupMode.COLD,
setupBlock = {
// Start the app
pressHome()
startActivityAndWait()
},
) {
// ... measureBlock
}
```
### With Macrobenchmark v1.2.0-alpha15:
```kotlin
private fun scrollFeed(compilationMode: CompilationMode) = benchmarkRule.measureRepeated(
// ...
startupMode = StartupMode.COLD,
setupBlock = {
// Start the app
pressHome()
startActivityAndWait()
},
) {
startActivityAndWait()
// ... measureBlock
}
```
`StartupMode.HOT` & `StartupMode.WARM` do not have any changes and work as expected.
### Relevant logcat output
```shell
java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.Object androidx.test.uiautomator.UiObject2.wait(androidx.test.uiautomator.UiObject2Condition, long)' on a null object reference
at com.google.samples.apps.nowinandroid.foryou.ForYouActionsKt.forYouWaitForContent(ForYouActions.kt:32)
at com.google.samples.apps.nowinandroid.foryou.ScrollForYouFeedBenchmark$scrollFeed$2.invoke(ScrollForYouFeedBenchmark.kt:52)
at com.google.samples.apps.nowinandroid.foryou.ScrollForYouFeedBenchmark$scrollFeed$2.invoke(ScrollForYouFeedBenchmark.kt:40)
at androidx.benchmark.macro.MacrobenchmarkKt$macrobenchmark$measurements$1$1$tracePath$1.invoke(Macrobenchmark.kt:260)
at androidx.benchmark.macro.MacrobenchmarkKt$macrobenchmark$measurements$1$1$tracePath$1.invoke(Macrobenchmark.kt:218)
at androidx.benchmark.perfetto.PerfettoCaptureWrapper.record(PerfettoCaptureWrapper.kt:129)
at androidx.benchmark.perfetto.PerfettoCaptureWrapper.record$default(PerfettoCaptureWrapper.kt:93)
at androidx.benchmark.macro.MacrobenchmarkKt$macrobenchmark$measurements$1.invoke(Macrobenchmark.kt:218)
at androidx.benchmark.macro.MacrobenchmarkKt$macrobenchmark$measurements$1.invoke(Macrobenchmark.kt:203)
at androidx.benchmark.perfetto.PerfettoTraceProcessor$Companion.runServer(PerfettoTraceProcessor.kt:105)
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmark(Macrobenchmark.kt:203)
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmarkWithStartupMode(Macrobenchmark.kt:388)
at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated(MacrobenchmarkRule.kt:107)
at com.google.samples.apps.nowinandroid.foryou.ScrollForYouFeedBenchmark.scrollFeed(ScrollForYouFeedBenchmark.kt:40)
at com.google.samples.apps.nowinandroid.foryou.ScrollForYouFeedBenchmark.scrollFeedCompilationNone(ScrollForYouFeedBenchmark.kt:35)
```
### Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Contributor guide
Assessment
This issue has not been assessed yet.