android / android/android-test

null cannot be used as parameter value in parameterized tests

Open
#837 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.2k
Forks
342
Avg merge
11h 29m
Merged PRs (30d)
2

Description

**Describe the bug**
Using `null` as a parameter value for parameterized tests does not work properly. The exact errors vary - sometimes it appears to get confused about the type of `null`:
```
java.lang.IllegalArgumentException: method com.example.BrokenTest. argument 1 has type java.lang.Integer, got java.lang.String
at java.lang.reflect.Constructor.newInstance(Native Method)
at org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTestUsingConstructorInjection(BlockJUnit4ClassRunnerWithParameters.java:56)
at org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTest(BlockJUnit4ClassRunnerWithParameters.java:46)
at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:260)
...
```

Other times it seems to get very confused about the class name of the test:
```
java.lang.ClassNotFoundException: Invalid name: positions=[0.0])]
at java.lang.Class.classForName(Native Method)
at java.lang.Class.forName(Class.java:324)
at androidx.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:72)
at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:105)
at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:804)
at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:575)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:393)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879)
```

(In this case, `positions=[0.0])` is the tail end of a `toString` for one of the other non-null parameter values.)

**To Reproduce**
The first case can be easily reproduced:
```kotlin
import org.junit.Test
import org.junit.runner.RunWith
import org.junit.runners.Parameterized
import kotlin.test.assertEquals

@RunWith(Parameterized::class)
class BrokenTest(private val testValue: Int?) {

@Test fun test() {
if (testValue != null) {
assertEquals(0, testValue % 2)
}
}

companion object {

@[JvmStatic Parameterized.Parameters(name = "{0}")]
fun parameters(): List = listOf(null, 0, 2, 4)
}
}

@RunWith(Parameterized::class)
class WorkingTest(private val testValue: Int?) {

@Test fun test() {
if (testValue != null) {
assertEquals(0, testValue % 2)
}
}

companion object {

@[JvmStatic Parameterized.Parameters(name = "{0}")]
fun parameters(): List = listOf(0, 2, 4)
}
}
```

I'm not yet 100% sure what causes the second but can investigate a minimal repro sample if needed (especially if that issue appears to be unrelated to the first).

**Expected behavior**
The above tests should all pass, as they do when run locally.

**Details (please complete the following information):**
> Have you tested on the latest Flank snapshot?

No

> Post the output of ` flank --version`.

20.09.3

Link: https://github.com/Flank/flank/issues/1374

We tracked this issue down to a limitation in orchestrator itself.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing BrokenTest and WorkingTest with the JUnit Parameterized runner, then read BlockJUnit4ClassRunnerWithParameters and the parameter-name handling shown in the report. Compare the null case with the non-null cases and check the linked Flank issue, where the limitation was tracked to the orchestrator; done means parameterized tests accept null values without the reported type or class-name errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kotlin
Domain
mobile, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.