allure-framework / allure-framework/allure-js

[allure playwright] Steps of skipped tests shouldn't be marked as failed

Open
#1,369 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
281
Forks
137
Avg merge
1d 11h
Merged PRs (30d)
13

Description

**Describe the bug**
Allure reporter results marks steps of skipped tests cases as failures instead of skipped.

**To Reproduce**
Steps to reproduce the behavior:
1. `test.skip()` a suite.
2. Run the tests, having allure as reporter
3. Check allure-results folder for test results
4. See that steps of skipped have status "failed" (see screenshot for details)

**Expected behavior**
In allure results, steps of skipped tests shouldn't be marked as failed. Instead the status should be "skipped", as the parent is.

**Screenshots**

Image

**Desktop (please complete the following information):**
- OS: win11
- Browser: chrome
- Version: 140
- Deps
"@playwright/test": "1.56.1",
"allure-playwright": 3.4.1
"playwright": "1.56.1"

**Additional context**
Seems it's important where the `test.skip()` is placed -> if it's placed not in `before` hook it seems to be ok, steps are not loaded.

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with a skipped Playwright suite and test.skip() in a before hook, then inspect the generated allure-results files and the step statuses. Done means steps belonging to skipped tests are reported as "skipped" rather than "failed", while the parent test remains skipped.

Written by the indexing model from the issue text.

Assessment

Tech stack
playwright, typescript
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.