allure-framework / allure-framework/allure3

Allure 3 report statistics and UI are incorrect on Windows for parallel runs with retries

Open
#616 0 comments 0 reactions 1 assignee Claimed by @formaceft-93 View on GitHub
Dominant language
HTML
Stars
401
Forks
58
Avg merge
2d 20h
Merged PRs (30d)
34

Description

**Describe the bug**
-----
Allure 3 can generate an incorrect report on Windows when the input allure-results directory contains parallel/xdist-produced result files that share the same raw uuid(due to reruns) which were run parallely.

In this case, the broken test result exists in allure-results, but the generated Allure 3 report does not show the broken test in the UI, and summary.json/widgets/statistic.json do not include the broken count.

When multiple retry result files have the same uuid, they receive the same internal id and overwrite each other in the store. Since directory iteration order differs between Windows and Linux, the final visible result can become OS-dependent.
**The same allure-results can produce the expected report on Linux
but an incorrect report on Windows.**

**To Reproduce**
-----
Python - 3.12.10
Allure - 3.6.2
Node - 24.13.1
WINDOWS 11
Running with pytest-xdist and pytest-rerunfailures

Steps to reproduce the behavior:
1. Clone https://github.com/legion-codes/playwright-pytest-bdd-demo.git
2. `uv venv` and `uv sync` to setup
2.1 `playwright install chromium`
3. `pytest -n 4 --reruns 5` (For my case. Issue visible in lower reruns as well)
4. 1 failure in `tests/steps/test_practice_test_automation_table_steps.py::test_filter_beginner_python_courses_with_many_enrollments`
5. `allure serve reports/allure-results` or `allure generate reports\allure-results -o reports\allure-report --open` to see the report build and wrong count in summary.json

This issue is not present if I use allure2 with JAVA in WINDOWS

**Expected behavior**
Allure 3 should generate the same report from the same allure-results directory on Windows and Linux.

If a retried test ultimately has a visible latest attempt with status broken, the generated report should show that test as broken in the UI, and the generated statistics should include the broken count, for example:

```json
"total": 26,
"passed": 25,
"retries": 1,
"broken": 1
```

**Screenshots**

Expected Correct Behaviour (Working in LINUX)
-----
Image

Current Behaviour In Windows
-----

Image

**Desktop (please complete the following information):**
- OS: WINDOWS 11
- Browser Chrome Version 147.0.7727.138
- Version 3.6.2

**Additional context**
In my reproduction, making the internal result id unique per result file, for example by including context.metadata.originalFileName, makes the broken test appear correctly in the report.

```id: md5(${raw.uuid || randomUUID()}:${context.metadata?.originalFileName ?? ""})```

context.metadata.originalFileName is already provided by the Allure2 reader here:
```ts
await visitor.visitTestResult(dest, { readerId, metadata: { originalFileName } });
```

I will wait for further Comments and discussion. I can also start by adding a fix for this issue ... if thought process align.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.