[jUnit formatter] Skipped steps are considered as errors
- Dominant language
- Go
- Stars
- 2.7k
- Forks
- 280
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 2
Description
We noticed that when a step fails in an scenario, subsequent steps are marked as skipped, which is OK imho.
When processing a test case (a scenario), all the steps that are skipped are included as errors (although having the `skipped` type). And because they are marked as error XML elements, we are not able to identify the root cause of why the scenario failed.
But why? Because the CI (Jenkins) provides a higher ranking to errors than to failures, causing the UI to display the fist step error instead of the first failure, which is misleading. I.e.:
```xml
```

Thankfully we store the XML file and then we were able to read the real error, hid by the first error.
With the above XML file, I'd expect the UI to display the failure step. In fact, removing all errors simply keeping failures makes the UI to display the proper error message.
Note: we are aware that, potentially, it could be an issue in the Jenkins side where errors are overriding the real failure, but wanted to share here our expectations first.
Contributor guide
Research direction
Start by locating the jUnit formatter and the code that serializes scenario steps into the shown XML. Reproduce a scenario where a failed step causes later steps to be skipped, then inspect the generated testcase elements. Done means skipped steps are not emitted as error elements while the original failure remains visible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100