spockframework / spockframework/spock

IGlobalExtension start() and stop() are run excess amount of times

Open
#1,240 16 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

3rd-party: Maven Surefire
Dominant language
Java
Stars
3.6k
Forks
483
PR merge metrics
No merged PRs in 30d

Description

Issue description

IGlobalExtension#start() and stop() are run during SpockEngine test collection even if there are no test classes for Spock to collect. This leads to start() and stop() being run even if there are no Spock tests to run. For example, if you run JUnit tests. This changed with Spock 2.0. Spock 1.3 doesn't exhibit this.
This happens in due to this chain org.spockframework.runtime.SpockEngine#discover -> org.spockframework.runtime.RunContext#get -> org.spockframework.runtime.RunContext#start . Discover might not return anything for Spock to run, but extensions will be fire any way.

How to reproduce

Instead of posting huge snippets of code here, here's the repo with minimalistic Maven project to illustrate this issue:
https://github.com/Net-burst/spock-junit5-runner
I added console output for start(), stop() and visitSpec(). Please look at when Spock extension start, Spock extension stop and Spock extension start visitSpec appear.

In Maven scenarios this leads to extension code being executed needlessly several times:

  • maven test (only JUnit classes are included in surefire plugin):
    • Spock extension start
    • Spock extension stop
    • JUnit tests
    • Spock extension stop
  • maven verify (only Spock specifications are included in the failsafe plugin):
    • same as maven test during surefire phase
    • Spock extension start
    • Spock extension visitSpec FirstSpec
    • Spock extension visitSpec SecondSpec
    • Spock extension visitSpec FirstSpec
    • Spock extension visitSpec SecondSpec
    • Spock tests
    • Spock extension stop
    • Spock extension stop

So, apart from start() being executed 2 times (first for the surefire plugin, when there are no collected Spock specifications and them for the failsafe plugin), stop() is also being executed 2 times. Also IDK why is visitSpec() executed an excess amount of times.

Additional Environment information

Java: Coretto 11.0.8
Spock: 2.0-M4-groovy-3.0
Groovy: 3.0.6
IDEA: 2020.2.4 Ultimate
OS: Linux (Ubuntu 20.04) / Windows 10

Java/JDK

openjdk 11.0.8 2020-07-14 LTS
OpenJDK Runtime Environment Corretto-11.0.8.10.1 (build 11.0.8+10-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.8.10.1 (build 11.0.8+10-LTS, mixed mode)

Groovy version

Groovy 3.0.6 running from IDEA as Maven dependency

Build tool version

Apache Maven
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Users\NetBurst\AppData\Local\JetBrains\Toolbox\apps\IDEA-U\ch-0\202.8194.7\plugins\maven\lib\maven3
Java version: 11.0.8, vendor: Amazon.com Inc., runtime: C:\Users\NetBurst\.jdks\corretto-11.0.8
Default locale: en_US, platform encoding: UTF-8
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

Operating System

Tested on both Ubuntu 20.04 and Windows 10

IDE

IDEA 2020.2.4 Ultimate

Build-tool dependencies used

Apache Maven
    <dependencies>
        <dependency>
            <groupId>org.spockframework</groupId>
            <artifactId>spock-core</artifactId>
            <version>2.0-M4-groovy-3.0</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.7.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.7.0</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.gmavenplus</groupId>
                <artifactId>gmavenplus-plugin</artifactId>
                <version>1.11.0</version>
                <configuration>
                    <parallelParsing>true</parallelParsing>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>compileTests</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.2</version>
                <configuration>
                    <includes>**/*Test.java</includes>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.22.2</version>
                <configuration>
                    <includes>**/*Spec.java</includes>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the behavior with the linked spock-junit5-runner Maven project, then trace org.spockframework.runtime.SpockEngine#discover through RunContext#get and RunContext#start. Compare runs with no Spock specifications and with Spock specifications; done means extensions are not started or stopped for empty discovery and are not invoked redundantly during collection.

Written by the indexing model from the issue text.

Assessment

Tech stack
groovy, java
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.