ASSERT-KTH / ASSERT-KTH/flacoco

Flacoco crashes when a test method name contains spaces

Open
#247 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
38
Forks
15
PR merge metrics
No merged PRs in 30d

Description

Some JVM languages (e.g., Kotlin) allow [method names to contain spaces](https://kotlinlang.org/docs/coding-conventions.html#names-for-test-methods). This convention is used in many projects, such as lettuce-core, which is a subject in the APR-Comp. It has [a test method](https://github.com/lettuce-io/lettuce-core/blob/main/src/test/kotlin/io/lettuce/core/ScanFlowIntegrationTests.kt#L50) that looks like this:

```kotlin
@Test
fun `should scan iteratively`() = runBlocking {
...
}
```

Flacoco will crash on such projects because it won't escape the spaces.

I forgot the exact error because I encountered this bug a few months ago and now I don't have a Kotlin environment at hand, but I still remember what's going on in the code:
1. Flacoco calls the `runOnlineCoveredTestResultPerTestMethods` method in test-runner with a list of collected test method names [(code)](https://github.com/ASSERT-KTH/flacoco/blob/master/src/main/java/fr/spoonlabs/flacoco/core/coverage/framework/JUnit4Strategy.java#L25).
2. `runOnlineCoveredTestResultPerTestMethods` will then concatenate all test method names with a space [(code)](https://github.com/STAMP-project/test-runner/blob/463c821c7d5d9ab947ccd24d6dd0c8a7fda7f1ec/src/main/java/eu/stamp_project/testrunner/EntryPoint.java#L718),
3. ... and then transform it into a command-line argument and call `runGivenCommandLine` [(code)](https://github.com/STAMP-project/test-runner/blob/463c821c7d5d9ab947ccd24d6dd0c8a7fda7f1ec/src/main/java/eu/stamp_project/testrunner/EntryPoint.java#L729).
4. `runGivenCommandLine` will then spawn a new Java process with the given argument [(code)](https://github.com/STAMP-project/test-runner/blob/463c821c7d5d9ab947ccd24d6dd0c8a7fda7f1ec/src/main/java/eu/stamp_project/testrunner/EntryPoint.java#L820).

Therefore, if a test method name contains a space, it will be parsed as two arguments, crashing the command-line parser.

Maybe you can reproduce this issue and forward it to test-runner if necessary. Thanks again for your great project!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in flacoco's JUnit4Strategy.java at runOnlineCoveredTestResultPerTestMethods, then trace the referenced test-runner EntryPoint.java methods runOnlineCoveredTestResultPerTestMethods and runGivenCommandLine. Reproduce with a Kotlin test method containing spaces, and verify that the test-runner receives the method name as one test identifier rather than split command-line arguments.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kotlin
Domain
testing-qa, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.