allure-framework / allure-framework/allure3
Allure 3 report statistics and UI are incorrect on Windows for parallel runs with retries
- Dominant language
- HTML
- Stars
- 401
- Forks
- 58
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 34
Description
**Describe the bug**
-----
Allure 3 can generate an incorrect report on Windows when the input allure-results directory contains parallel/xdist-produced result files that share the same raw uuid(due to reruns) which were run parallely.
In this case, the broken test result exists in allure-results, but the generated Allure 3 report does not show the broken test in the UI, and summary.json/widgets/statistic.json do not include the broken count.
When multiple retry result files have the same uuid, they receive the same internal id and overwrite each other in the store. Since directory iteration order differs between Windows and Linux, the final visible result can become OS-dependent.
**The same allure-results can produce the expected report on Linux
but an incorrect report on Windows.**
**To Reproduce**
-----
Python - 3.12.10
Allure - 3.6.2
Node - 24.13.1
WINDOWS 11
Running with pytest-xdist and pytest-rerunfailures
Steps to reproduce the behavior:
1. Clone https://github.com/legion-codes/playwright-pytest-bdd-demo.git
2. `uv venv` and `uv sync` to setup
2.1 `playwright install chromium`
3. `pytest -n 4 --reruns 5` (For my case. Issue visible in lower reruns as well)
4. 1 failure in `tests/steps/test_practice_test_automation_table_steps.py::test_filter_beginner_python_courses_with_many_enrollments`
5. `allure serve reports/allure-results` or `allure generate reports\allure-results -o reports\allure-report --open` to see the report build and wrong count in summary.json
This issue is not present if I use allure2 with JAVA in WINDOWS
**Expected behavior**
Allure 3 should generate the same report from the same allure-results directory on Windows and Linux.
If a retried test ultimately has a visible latest attempt with status broken, the generated report should show that test as broken in the UI, and the generated statistics should include the broken count, for example:
```json
"total": 26,
"passed": 25,
"retries": 1,
"broken": 1
```
**Screenshots**
Expected Correct Behaviour (Working in LINUX)
-----
Current Behaviour In Windows
-----
**Desktop (please complete the following information):**
- OS: WINDOWS 11
- Browser Chrome Version 147.0.7727.138
- Version 3.6.2
**Additional context**
In my reproduction, making the internal result id unique per result file, for example by including context.metadata.originalFileName, makes the broken test appear correctly in the report.
```id: md5(${raw.uuid || randomUUID()}:${context.metadata?.originalFileName ?? ""})```
context.metadata.originalFileName is already provided by the Allure2 reader here:
```ts
await visitor.visitTestResult(dest, { readerId, metadata: { originalFileName } });
```
I will wait for further Comments and discussion. I can also start by adding a fix for this issue ... if thought process align.
Contributor guide
Assessment
This issue has not been assessed yet.