allure-framework / allure-framework/allure-csharp
Exceptions in setup methods do not generate results
- Dominant language
- C#
- Stars
- 125
- Forks
- 76
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 18
Description
#### I'm submitting a ...
- [x] bug report
- [ ] feature request
- [ ] support request => Please do not submit support request here, see note at the top of this template.
#### What is the current behavior?
If something fails in a OneTimeSetup or Setup decorated method there are no results generated for the tests inside the class
#### If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
Code Example
```
using NUnit.Allure.Core;
namespace allure;
[TestFixture]
[AllureNUnit]
public class SetupError
{
[OneTimeSetUp]
public void Setup()
{
Console.Out.WriteLine("SetupStep");
throw new Exception("Exception");
}
[Test]
public void Test1()
{
Console.Out.WriteLine("Test1");
}
}
```
#### What is the expected behavior?
Test should be logged as skipped
#### What is the motivation / use case for changing the behavior?
Tests that are failed because of failed setup steps are completely missing from the report which can cause them to be missed completely
#### Please tell us about your environment:
- Allure version: 2.1.0
- Test framework: NUnit 3.13.3, Nunit3TestAdapter 4.4.2
- Allure adaptor: Allure.NUnit 2.9.5-preview1
- Generate report using: Allure CLI
#### Other information
This happens probably because of #325
Contributor guide
Research direction
Start with the provided NUnit reproduction using a failing OneTimeSetUp method and inspect how Allure.NUnit handles setup exceptions and test result generation. Confirm the behavior with NUnit 3.13.3 and Allure.NUnit 2.9.5-preview1; done means the affected test is represented in the Allure results as skipped rather than missing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100