android / android/android-test
Frequent 'No tests found' when sharding tests
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 342
- Avg merge
- 11h 29m
- Merged PRs (30d)
- 2
Description
### Description
When sharding tests I get frequent (and seemingly random) 'No tests found' errors which fail the build.
See this PR: https://github.com/google/accompanist/pull/463 for an example.
### Steps to Reproduce
I'm not 100% sure what the repro is. Having a small number of tests in the module seems to be the trigger.
I think this is because the library is relying on [hashCode being uniformly random](https://github.com/android/android-test/blob/9d25d24ec7abe5021672c0423a992e87b44eb261/runner/android_junit_runner/java/androidx/test/internal/runner/TestRequestBuilder.java#L338) over a small data-set, which isn't the case. If the module only has ~5 tests, and I'm sharding over 3 runs, the chance of at least 1 shard having 0 tests is quite high.
### Expected Results
Shards with no tests from a module don't fail the build.
### Actual Results
```
com.android.build.gradle.internal.testing.ConnectedDevice > No tests found.[test(AVD) - 5.1.1] FAILED
No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @Test annotations).
```
### AndroidX Test and Android OS Versions
[`1.3.0`](https://github.com/google/accompanist/blob/main/gradle/libs.versions.toml#L11)
### Link to a public git repo demonstrating the problem:
https://github.com/google/accompanist/pull/463
Contributor guide
Research direction
Start with the TestRequestBuilder.java implementation linked in the issue and the sharding behavior demonstrated by accompanist PR 463. Reproduce the small-module, multi-shard case and verify that shards receiving no tests no longer fail the build.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- mobile-dev, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100