android / android/android-test
null cannot be used as parameter value in parameterized tests
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 342
- Avg merge
- 11h 29m
- Merged PRs (30d)
- 2
Description
**Describe the bug**
Using `null` as a parameter value for parameterized tests does not work properly. The exact errors vary - sometimes it appears to get confused about the type of `null`:
```
java.lang.IllegalArgumentException: method com.example.BrokenTest. argument 1 has type java.lang.Integer, got java.lang.String
at java.lang.reflect.Constructor.newInstance(Native Method)
at org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTestUsingConstructorInjection(BlockJUnit4ClassRunnerWithParameters.java:56)
at org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTest(BlockJUnit4ClassRunnerWithParameters.java:46)
at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:260)
...
```
Other times it seems to get very confused about the class name of the test:
```
java.lang.ClassNotFoundException: Invalid name: positions=[0.0])]
at java.lang.Class.classForName(Native Method)
at java.lang.Class.forName(Class.java:324)
at androidx.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:72)
at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:105)
at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:804)
at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:575)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:393)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879)
```
(In this case, `positions=[0.0])` is the tail end of a `toString` for one of the other non-null parameter values.)
**To Reproduce**
The first case can be easily reproduced:
```kotlin
import org.junit.Test
import org.junit.runner.RunWith
import org.junit.runners.Parameterized
import kotlin.test.assertEquals
@RunWith(Parameterized::class)
class BrokenTest(private val testValue: Int?) {
@Test fun test() {
if (testValue != null) {
assertEquals(0, testValue % 2)
}
}
companion object {
@[JvmStatic Parameterized.Parameters(name = "{0}")]
fun parameters(): List = listOf(null, 0, 2, 4)
}
}
@RunWith(Parameterized::class)
class WorkingTest(private val testValue: Int?) {
@Test fun test() {
if (testValue != null) {
assertEquals(0, testValue % 2)
}
}
companion object {
@[JvmStatic Parameterized.Parameters(name = "{0}")]
fun parameters(): List = listOf(0, 2, 4)
}
}
```
I'm not yet 100% sure what causes the second but can investigate a minimal repro sample if needed (especially if that issue appears to be unrelated to the first).
**Expected behavior**
The above tests should all pass, as they do when run locally.
**Details (please complete the following information):**
> Have you tested on the latest Flank snapshot?
No
> Post the output of ` flank --version`.
20.09.3
Link: https://github.com/Flank/flank/issues/1374
We tracked this issue down to a limitation in orchestrator itself.
Contributor guide
Research direction
Start by reproducing BrokenTest and WorkingTest with the JUnit Parameterized runner, then read BlockJUnit4ClassRunnerWithParameters and the parameter-name handling shown in the report. Compare the null case with the non-null cases and check the linked Flank issue, where the limitation was tracked to the orchestrator; done means parameterized tests accept null values without the reported type or class-name errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kotlin
- Domain
- mobile, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100