allure-framework / allure-framework/allure-csharp
The test is marked as passed although it raises a TechTalk.SpecFlow.BindingException
- Dominant language
- C#
- Stars
- 125
- Forks
- 76
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 18
Description
[//]: # (
. Note: for support questions, please use Stackoverflow or Gitter**.
. This repository's issues are reserved for feature requests and bug reports.
.
. In case of any problems with Allure Jenkins plugin** please use the following repository
. to create an issue: https://github.com/jenkinsci/allure-plugin/issues
.
. Make sure you have a clear name for your issue. The name should start with a capital
. letter and no dot is required in the end of the sentence. An example of good issue names:
.
. - The report is broken in IE11
. - Add an ability to disable default plugins
. - Support emoji in test descriptions
)
#### I'm submitting a ...
- [x] bug report
- [ ] feature request
- [ ] support request => Please do not submit support request here, see note at the top of this template.
#### What is the current behavior?
We execute a test and it raises and **TechTalk.SpecFlow.BindingException** but it isn't catched by the AllureLifCycle object and it is marked as passed.
#### If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
We are using Specflow 2 with NUnit 3.
In order to reproduce the scenario is: you have to decored two step methods with the same specflow test. Beacause we have did it in our test suite by error.
[When(@"I save and back")]
public void WhenIGoBackCancelingAnyChanges()
{
...
}
[When(@"I save and back")]
public void WhenISaveAndBack()
{
...
}
When you do this by error, NUnit catches the TechTalk.SpecFlow.BindingException and only it's logged by NUnit. The ScenarioContext don't have logged the exception and specflow don't excute the BeforeStep hook.
In allure results files the failing step has the status none:
"name": "And I save and back",
"status": "none",
And the next steps are marked as skipped. But the information about the fail is in the TestResult object of NUnit.
#### What is the expected behavior?
We expect that the test was marked as **Broken**.
#### What is the motivation / use case for changing the behavior?
#### Please tell us about your environment:
- Allure version: 2.4.2.4
- Test framework: Specflow 2 and NUnit 3
- Allure adaptor: allure-Specflow@2.4.2.4
- Generate report using: Allure-Docker-Service
#### Other information
[//]: # (
. e.g. detailed explanation, stacktraces, related issues, suggestions
. how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc
)
Contributor guide
Research direction
Trace how the AllureLifecycle object receives NUnit TestResult data for SpecFlow 2 scenarios, focusing on the ScenarioContext and BeforeStep hook path described. Reproduce the duplicate [When] binding case, then verify the generated Allure result marks the failing step and test as Broken rather than status none or passed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100