allure-framework / allure-framework/allure-python

[allure-behave] Support @flaky tag for scenarios (to mark them as flaky in the allure report)

Open
#305 2 comments 0 reactions 0 assignees View on GitHub
task:new feature theme:behave
Dominant language
Python
Stars
814
Forks
260
PR merge metrics
No merged PRs in 30d

Description

#### I'm submitting a ...
- [ ] bug report
- [x] feature request
- [ ] support request => Please do not submit support request here, see note at the top of this template.

#### What is the current behavior?
In an allure report based on the output of the `allure-behave` formatter, a `@flaky` tag is simply shown in the list of all scenario tags.
But the scenario is not marked as flaky or special in any way.

#### If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
Use the allure-behave formatter for testruns with scenarios containing a `@flaky` tag and view the corresponding allure test report.

#### What is the expected behavior?
It would be nice if a scenario tagged as flaky was marked as flaky in the json file produced by the `allure-behave` formatter (allure's flaky attribute set to `True`) - and thus shown as flaky test in a resulting allure test report.

#### What is the motivation / use case for changing the behavior?
Scenarios tagged as flaky could be more easily spotted in the overall allure test report.

Allure does support marking tests as flaky:
https://docs.qameta.io/allure/#_flaky_tests

Tagging a scenario or feature as flaky is apparently already supported by `allure-cucumber-jvm`:
https://docs.qameta.io/allure/#_test_markers

#### Please tell us about your environment:

- allure-behave 2.5.1
- allure-python-commons 2.5.1
- allure 2.3-SNAPSHOT

#### Other information

[//]: # (
. e.g. detailed explanation, stacktraces, related issues, suggestions
. how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc
)

Contributor guide

Open the contributing guide

Research direction

Start at the allure-behave formatter and trace how scenario tags are converted into the JSON result. Run formatter output for a scenario with @flaky, then verify the generated JSON sets Allure's flaky attribute and the resulting report marks the scenario as flaky.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.