allure-framework / allure-framework/allure-csharp
Distinguish between failed and broken tests when using a 3rd party assertion library with Allure.NUnit and Allure.Xunit
- Dominant language
- C#
- Stars
- 125
- Forks
- 76
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 18
Description
[//]: # (
. Note: for support questions, please create a discussion at https://github.com/orgs/allure-framework/discussions.
. This repository's issues are reserved for feature requests and bug reports.
.
. Make sure you have a clear name for your issue. The name should start with a capital
. letter and no dot is required in the end of the sentence. An example of good issue names:
.
. - The report is broken in IE11
. - Add an ability to disable default plugins
. - Support emoji in test descriptions
)
#### I'm submitting a ...
- [X] bug report
- [ ] feature request
- [ ] support request => Please do not submit support request here, see note at the top of this template.
#### What is the current behavior?
The test case failed using fluentassertions or Playwright are shown as broken instead of failed
#### If the current behavior is a bug, please provide the steps to reproduce and, if possible, a minimal demo of the problem
Create a simple test using FluentAssertions or Playwright dotnet, when you run the tests the resutl is shown as broken

#### What is the expected behavior?
The test should be failed if I have configured the list of exceptions (failExceptions).
#### What is the motivation / use case for changing the behavior?
#### Please tell us about your environment:
- OS Windows
- .NET framework .NET 8
- Test framework: nunit@4.0.0
- Allure adapter: allure-nunit@2.12.0
#### Other information
I was taking a look to the AllureNUnitHelper and I was able to see that the IsBroken function is waiting elements in result.Assertions list but when you use FluentAssertions or Playwright this list is not update so the tests are always mark as broken.

Contributor guide
Research direction
Start with AllureNUnitHelper.IsBroken and inspect how result.Assertions is used when running a FluentAssertions or Playwright test. Reproduce the behavior with the stated .NET 8 and NUnit setup, then verify that exceptions listed in failExceptions are reported as failed rather than broken.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, playwright
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100