allure-framework / allure-framework/allure-python

In Scenario Outline of Behave, Allure reports include Skipped test even when uisng --no-skipped or show_skipped = false

Open
#432 3 comments 0 reactions 0 assignees View on GitHub
bug theme:behave
Dominant language
Python
Stars
814
Forks
260
PR merge metrics
No merged PRs in 30d

Description

I'm submitting a ...
- [.] bug report
- [ ] feature request

What is the current behavior?
In **Scenario Outline** of Behave, Skipped issues appear in Allure Results even if Skipped --no-skipped flag is set or behave.ini is set to show_skipped = false

Steps:
1) I have written 2 scenario outlines. Each scenario outline loops through 4 kits(C, Java, .Net, .Net core) for 2 different examples.
2) In first scenario outline, after first scenario ie, C, I have skipped remaining 3 scenarios of Java, .Net, .Net core)
3) When it rolls around second scenario outline, in Allure reports I can see the skipped scenario results of first scenario outline which should not be (highlighted with yellow color in one of the attachments).
4) behave is skipped the scenarios properly
![tests feature](https://user-images.githubusercontent.com/55695053/65425521-c913f880-de2b-11e9-89b5-4f55adf2ec03.JPG)
![StepDefs](https://user-images.githubusercontent.com/55695053/65424676-bd273700-de29-11e9-8291-d869664a94a4.JPG)
![allure report snap 1](https://user-images.githubusercontent.com/55695053/65424677-bd273700-de29-11e9-8572-970786ec1efc.JPG)
![Behave_ouput_1](https://user-images.githubusercontent.com/55695053/65424678-bdbfcd80-de29-11e9-9ba9-938a424ccacf.JPG)
![Behave_ouput_2](https://user-images.githubusercontent.com/55695053/65424679-bdbfcd80-de29-11e9-931b-83b3d8145406.JPG)

What is the expected behavior?
The skipped scenario results of previous scenario outline should not appear in next scenario outline results

Please tell us about your environment:
- Allure-version: allure-behave@2.8.5
- Test framework: behave@1.2.6

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the report with behave@1.2.6 and allure-behave@2.8.5, using two scenario outlines and the --no-skipped or show_skipped = false setting. Inspect the generated Allure results to compare skipped examples from the first outline with the expected report, where those skipped scenarios do not appear.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.