JetBrains-Research / JetBrains-Research/TestSpark
[Prompt-scarcity] List of cases when the prompt is not sufficient (feel free to add)
- Dominant language
- Kotlin
- Stars
- 88
- Forks
- 24
- PR merge metrics
- No merged PRs in 30d
Description
### 1. Not adding enum properties to the prompt when the enum is the part of function parameters
We want to test the following class inside the TestSpark
```
org.jetbrains.research.testspark.core.test.strategies.JUnitTestSuiteParserStrategy
```
the prompt looks like this
--------------------------------------------------
Generate unit tests in kotlin for org.jetbrains.research.testspark.core.test.strategies.JUnitTestSuiteParserStrategy to achieve 100% line coverage for this class.
Dont use @Before and @After test methods.
Make tests as atomic as possible.
All tests should be for JUnit 5.
In case of mocking, use Mockito 5. But, do not use mocking for all tests.
Name all methods according to the template - [MethodUnderTest][Scenario]Test, and use only English letters.
The source code of class under test is as follows:
```
package org.jetbrains.research.testspark.core.test.strategies
import org.jetbrains.research.testspark.core.data.JUnitVersion
import org.jetbrains.research.testspark.core.test.TestCaseParseResult
import org.jetbrains.research.testspark.core.test.TestSuiteParserStrategy
import org.jetbrains.research.testspark.core.test.data.TestCaseGeneratedByLLM
import org.jetbrains.research.testspark.core.test.data.TestLine
import org.jetbrains.research.testspark.core.test.data.TestLineType
import org.jetbrains.research.testspark.core.test.data.TestSuiteGeneratedByLLM
class JUnitTestSuiteParserStrategy : TestSuiteParserStrategy {
}
```
org.jetbrains.research.testspark.core.test.strategies.JUnitTestSuiteParserStrategy extends org.jetbrains.research.testspark.core.test.TestSuiteParserStrategy. The source code of org.jetbrains.research.testspark.core.test.TestSuiteParserStrategy is:
```
package org.jetbrains.research.testspark.core.test
import org.jetbrains.research.testspark.core.data.JUnitVersion
import org.jetbrains.research.testspark.core.test.data.TestSuiteGeneratedByLLM
interface TestSuiteParserStrategy {
fun parseTestSuite(
rawText: String,
junitVersion: JUnitVersion,
importPattern: Regex,
packageName: String,
testNamePattern: String,
): TestSuiteGeneratedByLLM?
}
```
Generate public classes and public methods. Response only a code with tests between `, do not provide any other text.
--------------------------------------------------
As you can notice, in this prompt we do not have any information about parameters in `JUnitVersion`, therefore generated tests will not be compilable
Contributor guide
Assessment
This issue has not been assessed yet.