AfterScenario Hook runs out of order vs Step hooks when there has been an error
- Dominant language
- Go
- Stars
- 2.7k
- Forks
- 280
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 2
Description
### 👓 What did you see?
We have a test in suite_context_test.go tht makes an assertion on what I consider the **wrong** behaviour.
name: "fail_then_undefined_fails_scenario", afterStepCnt: 2, beforeStepCnt: 2,
body: `
When step fails
Then step is undefined`,
log: `
>>>> Before suite
>> Before scenario "test"
Before step "step fails"
After step "step fails", error: oops, status: failed
<< After scenario "test", error: oops
Before step "step is undefined"
After step "step is undefined", error: step is undefined, status: undefined
<<<< After suite`,
},
Above we can see that the 'AfterScenario' hook is expected called **before the scenario is actually over** and the remaining steps have their BeforeStep/AfterStep hooks called.
### ✅ What did you expect to see?
I am expecting the hooks to properly nest.
We should not call the After Scenario until all scenarios have been processed.
I expect this...
>>>> Before suite
>> Before scenario "test"
Before step "step fails"
After step "step fails", error: oops, status: failed
Before step "step is undefined"
After step "step is undefined", error: step is undefined, status: undefined
<< After scenario "test", error: oops
<<<< After suite`,
### 📦 Which tool/library version are you using?
v0.14.1 and also main
### 🔬 How could we reproduce it?
Just inspect the test cases in TestTestSuite_Run in suite_contexr_test.go
I believe this logic I plain wrong.
### 📚 Any additional context?
Seems same since at least v0.13
The formatters like events/cucumber look ok - but the hook order is bogus.
Contributor guide
Research direction
Start with TestTestSuite_Run in suite_context_test.go and inspect the failing hook-order test and its logged sequence. Trace the scenario execution around the error, then update the behavior and assertions so the remaining step hooks run before AfterScenario and the expected log passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100