allure-framework / allure-framework/allure-js
cucumber.js + playwright + allure shows N-1 tests in a report
- Dominant language
- TypeScript
- Stars
- 281
- Forks
- 137
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 13
Description
For a long time, I have been facing the issue that allure report shows N-1 tests in a report (e.g. 99 out of 100), the last test case is lost somehow
At the same time, all other available reports show ok, xml and json files contain the data for all tests
Asked AI about that, it suggested that because allure writes files asynchronously, it may need more time to flush, but adding timeouts after all tests will not help
The test case that is lost is not the last test case. It is somewhere in the middle of the test run
I am using azure devops and one of the extensions in the marketplace to build and show a report
**To Reproduce**
Steps to reproduce the behavior:
Complete a lot of test cases and generate a report
**Expected behavior**
all test cases are shown
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: Windows 11
- Node 22, 24, Cucumber.js v12.x, allure-cucumberjs 3.4.1
- tests are running in Azure DevOps agents
**Additional context**
generating reports locally produces the same results, so the configuration of the plugin should be ok
I have the following format configuration in cucumber config
format: [
...common.format,
"allure-cucumberjs/reporter:allure",
'json:./reports/cucumber.json',
'html:./reports/cucumber-embedded.html',
'junit:./reports/cucumber.xml'
],
formatOptions: {
...common.formatOptions,
resultsDir: "allure-results"
},
Contributor guide
Research direction
Start with the reported Cucumber configuration, especially the allure-cucumberjs reporter and resultsDir set to allure-results. Reproduce the missing-test behavior locally with the listed Node, Cucumber.js, Playwright, and allure-cucumberjs versions, then compare the generated Allure results with cucumber.json and cucumber.xml. Done means all executed test cases appear in the Allure report.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, node.js, playwright, typescript
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100