allure-framework / allure-framework/allure-js
Newman: wrong allure-report after collection run
- Dominant language
- TypeScript
- Stars
- 281
- Forks
- 137
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 13
Description
**Describe the bug**
Test is passed in allure-report, but collection run result is failed
**To Reproduce**
Steps to reproduce the behavior:
1. Run any collection with expected failed result (e.g. endpoint is not accessible) without allure-report:
`newman run `
2. Take a look at result
3. Retry first step with allure-report:
`newman run -r allure`
4. See allure-report
**Expected behavior**
Allure-report has all tests failed
**Screenshots**


[allure-results.zip](https://github.com/allure-framework/allure-js/files/12411375/allure-results.zip)
**Desktop (please complete the following information):**
- OS: Windows 10 Pro
- Browser: Chrome
- Version: 116.0.5845.97
- Allure version: 2.23.1
**Smartphone (please complete the following information):**
Not applicable
**Additional context**
I didn't check mixed result with buth passed and failed tests
Contributor guide
Research direction
Start by reproducing the two Newman commands in the issue, first without the allure reporter and then with it, and compare the generated Allure report with the attached allure-results.zip. Trace how the failed collection result is represented in the Allure output. Done means failed collection tests are reported as failed in Allure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100