allure-framework / allure-framework/allure-python

with `pytest-rerunfailures`, reruns are not recognized

Open
#815 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
814
Forks
260
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
Running a Pytest Test-Suite with the pytest-rerunfailures plugin does not recognize reruns. Instead, the result of the last run test is shown. Previously failed tests can be found as duplicates in the timeline. The flaky marker is not applied.

**Expected behavior**
The result of the last run test is shown. A flaky marker is applied. I am able to filter for tests that were rerun.

**Screenshots**
Example Test first run (failed):
![failed test (first run)](https://user-images.githubusercontent.com/1344744/105320469-7d5f9380-5bc6-11eb-89a0-ab8522e23244.png)
Example Test second run via --reruns=5 (succeeded)
![succeeded test (second run)](https://user-images.githubusercontent.com/1344744/105320474-7e90c080-5bc6-11eb-9cdc-40fd50690a26.png)
No flaky marker for test:
![no flaky marker](https://user-images.githubusercontent.com/1344744/105320594-a41dca00-5bc6-11eb-8f5e-a519c002db4e.png)

**Environment (please complete the following information):**

| Allure version | 2.13.6 |
| Python | 3.6.12 |
| Test framework | pytest@6.1.2 |
| Allure adaptor | allure-pytest@2.8.31, pytest-rerunfailures@9.1.1 |
| Generate report using | https://github.com/simple-elf/allure-report-action |

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with the listed Python, pytest, allure-pytest, and pytest-rerunfailures versions, comparing the first failed run with the successful rerun. Trace how the rerun appears in the timeline and report, and consider the issue complete when the final result is shown once, a flaky marker is applied, and rerun tests can be filtered.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.