spockframework / spockframework/spock

Test failures coming from JUnit4 rules are incorrectly attributed to subsequent iterations of unrolled features

Open
#1,268 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
3.6k
Forks
483
PR merge metrics
No merged PRs in 30d

Description

Issue description

When JUnit4 rule throws while handling an unrolled feature iteration, the test failure is incorrectly attributed to the subsequent iteration. The outcome looks identical to https://github.com/spockframework/spock/issues/1267 and could point to the same root cause.

How to reproduce

import org.junit.Rule
import org.junit.rules.TestRule
import org.junit.runner.Description
import org.junit.runners.model.Statement
import spock.lang.Specification

class RuleSpec extends Specification {

    @Rule
    public SomeRule rule = new SomeRule()

    def "feature with rule"() {
        expect:
        true

        where:
        foo << ['a', 'b', 'c', 'd']
    }
}

class SomeRule implements TestRule {
    @Override
    Statement apply(Statement base, Description description) {
        return new Statement() {
            @Override
            void evaluate() throws Throwable {
                base.evaluate()
                throw new RuntimeException("failure in rule")
            }
        }
    }
}

Executing the above test will report the failures like this:
image

Additional Environment information

Build-tool dependencies used

testImplementation("org.spockframework:spock-core:2.0-M4-groovy-3.0")
testImplementation("org.spockframework:spock-junit4:2.0-M4-groovy-3.0")

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the supplied RuleSpec reproduction with spock-core and spock-junit4 2.0-M4-groovy-3.0, then compare the result with issue 1267. Trace how the JUnit4 rule failure is assigned across the unrolled iterations; done means each failure is attributed to the iteration where the rule throws.

Written by the indexing model from the issue text.

Assessment

Tech stack
groovy, java
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.