spockframework / spockframework/spock
IGlobalExtension start() and stop() are run excess amount of times
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 483
- PR merge metrics
- No merged PRs in 30d
Description
Issue description
IGlobalExtension#start() and stop() are run during SpockEngine test collection even if there are no test classes for Spock to collect. This leads to start() and stop() being run even if there are no Spock tests to run. For example, if you run JUnit tests. This changed with Spock 2.0. Spock 1.3 doesn't exhibit this.
This happens in due to this chain org.spockframework.runtime.SpockEngine#discover -> org.spockframework.runtime.RunContext#get -> org.spockframework.runtime.RunContext#start . Discover might not return anything for Spock to run, but extensions will be fire any way.
How to reproduce
Instead of posting huge snippets of code here, here's the repo with minimalistic Maven project to illustrate this issue:
https://github.com/Net-burst/spock-junit5-runner
I added console output for start(), stop() and visitSpec(). Please look at when Spock extension start, Spock extension stop and Spock extension start visitSpec appear.
In Maven scenarios this leads to extension code being executed needlessly several times:
- maven test (only JUnit classes are included in surefire plugin):
- Spock extension start
- Spock extension stop
- JUnit tests
- Spock extension stop
- maven verify (only Spock specifications are included in the failsafe plugin):
- same as maven test during surefire phase
- Spock extension start
- Spock extension visitSpec FirstSpec
- Spock extension visitSpec SecondSpec
- Spock extension visitSpec FirstSpec
- Spock extension visitSpec SecondSpec
- Spock tests
- Spock extension stop
- Spock extension stop
So, apart from start() being executed 2 times (first for the surefire plugin, when there are no collected Spock specifications and them for the failsafe plugin), stop() is also being executed 2 times. Also IDK why is visitSpec() executed an excess amount of times.
Additional Environment information
Java: Coretto 11.0.8
Spock: 2.0-M4-groovy-3.0
Groovy: 3.0.6
IDEA: 2020.2.4 Ultimate
OS: Linux (Ubuntu 20.04) / Windows 10
Java/JDK
openjdk 11.0.8 2020-07-14 LTS
OpenJDK Runtime Environment Corretto-11.0.8.10.1 (build 11.0.8+10-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.8.10.1 (build 11.0.8+10-LTS, mixed mode)
Groovy version
Groovy 3.0.6 running from IDEA as Maven dependency
Build tool version
Apache Maven
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Users\NetBurst\AppData\Local\JetBrains\Toolbox\apps\IDEA-U\ch-0\202.8194.7\plugins\maven\lib\maven3
Java version: 11.0.8, vendor: Amazon.com Inc., runtime: C:\Users\NetBurst\.jdks\corretto-11.0.8
Default locale: en_US, platform encoding: UTF-8
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
Operating System
Tested on both Ubuntu 20.04 and Windows 10
IDE
IDEA 2020.2.4 Ultimate
Build-tool dependencies used
Apache Maven
<dependencies>
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-core</artifactId>
<version>2.0-M4-groovy-3.0</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.7.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.7.0</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.11.0</version>
<configuration>
<parallelParsing>true</parallelParsing>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compileTests</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<includes>**/*Test.java</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<includes>**/*Spec.java</includes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the behavior with the linked spock-junit5-runner Maven project, then trace org.spockframework.runtime.SpockEngine#discover through RunContext#get and RunContext#start. Compare runs with no Spock specifications and with Spock specifications; done means extensions are not started or stopped for empty discovery and are not invoked redundantly during collection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- groovy, java
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100