allure-framework / allure-framework/allure-csharp

Exceptions in setup methods do not generate results

Open
#342 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C#
Stars
125
Forks
76
Avg merge
1d 2h
Merged PRs (30d)
18

Description

#### I'm submitting a ...
- [x] bug report
- [ ] feature request
- [ ] support request => Please do not submit support request here, see note at the top of this template.

#### What is the current behavior?
If something fails in a OneTimeSetup or Setup decorated method there are no results generated for the tests inside the class

#### If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
Code Example
```
using NUnit.Allure.Core;

namespace allure;

[TestFixture]
[AllureNUnit]
public class SetupError
{
[OneTimeSetUp]
public void Setup()
{
Console.Out.WriteLine("SetupStep");
throw new Exception("Exception");
}

[Test]
public void Test1()
{
Console.Out.WriteLine("Test1");
}
}
```

#### What is the expected behavior?
Test should be logged as skipped

#### What is the motivation / use case for changing the behavior?
Tests that are failed because of failed setup steps are completely missing from the report which can cause them to be missed completely

#### Please tell us about your environment:

- Allure version: 2.1.0
- Test framework: NUnit 3.13.3, Nunit3TestAdapter 4.4.2
- Allure adaptor: Allure.NUnit 2.9.5-preview1
- Generate report using: Allure CLI

#### Other information
This happens probably because of #325

Contributor guide

Open the contributing guide

Research direction

Start with the provided NUnit reproduction using a failing OneTimeSetUp method and inspect how Allure.NUnit handles setup exceptions and test result generation. Confirm the behavior with NUnit 3.13.3 and Allure.NUnit 2.9.5-preview1; done means the affected test is represented in the Allure results as skipped rather than missing.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.