element-hq / element-hq/element-android
Investigate integration test failures when all tests run together
- Dominant language
- Kotlin
- Stars
- 3.7k
- Forks
- 917
- Avg merge
- 8d 20h
- Merged PRs (30d)
- 1
Description
```
Mar 08, 2022 2:26:04 PM com.google.testing.platform.RunnerImpl$Companion summarizeAndLogTestResult
SEVERE: Execute org.matrix.android.sdk.internal.crypto.keysbackup.KeysBackupTest.testBackupWhenAnotherBackupWasCreated: FAILED
java.lang.AssertionError: java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at org.matrix.android.sdk.internal.crypto.keysbackup.KeysBackupTest.testBackupWhenAnotherBackupWasCreated(KeysBackupTest.kt:981)
java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at org.matrix.android.sdk.internal.crypto.keysbackup.KeysBackupTest.testBackupWhenAnotherBackupWasCreated(KeysBackupTest.kt:981)
Mar 08, 2022 2:26:04 PM com.google.testing.platform.RunnerImpl$Companion summarizeAndLogTestResult
SEVERE: Execute org.matrix.android.sdk.session.space.SpaceHierarchyTest.testParentRelation: FAILED
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java: java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at org.matrix.android.sdk.common.CommonTestHelper.await(CommonTestHelper.kt:378)
at org.matrix.android.sdk.common.CommonTestHelper.waitWithLatch(CommonTestHelper.kt:396)
at org.matrix.android.sdk.common.CommonTestHelper.waitWithLatch$default(CommonTestHelper.kt:391)
at org.matrix.android.sdk.session.space.SpaceHierarchyTest.createPrivateSpace(SpaceHierarchyTest.kt:410)
at org.matrix.android.sdk.session.space.SpaceHierarchyTest.testParentRelation(SpaceHierarchyTest.kt:491)
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at org.matrix.android.sdk.common.CommonTestHelper.await(CommonTestHelper.kt:378)
at org.matrix.android.sdk.common.CommonTestHelper.waitWithLatch(CommonTestHelper.kt:396)
at org.matrix.android.sdk.common.CommonTestHelper.waitWithLatch$default(CommonTestHelper.kt:391)
at org.matrix.android.sdk.session.space.SpaceHierarchyTest.createPrivateSpace(SpaceHierarchyTest.kt:410)
at org.matrix.android.sdk.session.space.SpaceHierarchyTest.testParentRelation(SpaceHierarchyTest.kt:491)
Mar 08, 2022 2:26:04 PM com.google.testing.platform.RunnerImpl$Companion summarizeAndLogTestResult
SEVERE: Execute org.matrix.android.sdk.session.space.SpaceHierarchyTest.testRootSpaces: FAILED
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java: java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at org.matrix.android.sdk.common.CommonTestHelper.await(CommonTestHelper.kt:378)
at org.matrix.android.sdk.common.CommonTestHelper.waitWithLatch(CommonTestHelper.kt:396)
at org.matrix.android.sdk.common.CommonTestHelper.waitWithLatch$default(CommonTestHelper.kt:391)
at org.matrix.android.sdk.session.space.SpaceHierarchyTest.createPublicSpace(SpaceHierarchyTest.kt:382)
at org.matrix.android.sdk.session.space.SpaceHierarchyTest.testRootSpaces(SpaceHierarchyTest.kt:445)
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at org.matrix.android.sdk.common.CommonTestHelper.await(CommonTestHelper.kt:378)
at org.matrix.android.sdk.common.CommonTestHelper.waitWithLatch(CommonTestHelper.kt:396)
at org.matrix.android.sdk.common.CommonTestHelper.waitWithLatch$default(CommonTestHelper.kt:391)
at org.matrix.android.sdk.session.space.SpaceHierarchyTest.createPublicSpace(SpaceHierarchyTest.kt:382)
at org.matrix.android.sdk.session.space.SpaceHierarchyTest.testRootSpaces(SpaceHierarchyTest.kt:445)
```
The above three failures occurred when running `:matrix-android-sdk:connectedDebugAndroidTest` with no package/class limits, in both CI and locally on my desktop.
The tests did not individually fail when running in five separate package-scoped limited runs, so there is some interaction between them (or perhaps between a previous test and them) that is causing this issue.
Contributor guide
Assessment
This issue has not been assessed yet.