allure-framework / allure-framework/allure-js

cucumber.js + playwright + allure shows N-1 tests in a report

Open
#1,387 2 comments 0 reactions 0 assignees View on GitHub
theme:cucumberjs theme:playwright
Dominant language
TypeScript
Stars
281
Forks
137
Avg merge
1d 11h
Merged PRs (30d)
13

Description

For a long time, I have been facing the issue that allure report shows N-1 tests in a report (e.g. 99 out of 100), the last test case is lost somehow
At the same time, all other available reports show ok, xml and json files contain the data for all tests

Asked AI about that, it suggested that because allure writes files asynchronously, it may need more time to flush, but adding timeouts after all tests will not help

The test case that is lost is not the last test case. It is somewhere in the middle of the test run

I am using azure devops and one of the extensions in the marketplace to build and show a report

**To Reproduce**
Steps to reproduce the behavior:
Complete a lot of test cases and generate a report

**Expected behavior**
all test cases are shown

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: Windows 11
- Node 22, 24, Cucumber.js v12.x, allure-cucumberjs 3.4.1
- tests are running in Azure DevOps agents

**Additional context**
generating reports locally produces the same results, so the configuration of the plugin should be ok

I have the following format configuration in cucumber config
format: [
...common.format,
"allure-cucumberjs/reporter:allure",
'json:./reports/cucumber.json',
'html:./reports/cucumber-embedded.html',
'junit:./reports/cucumber.xml'
],
formatOptions: {
...common.formatOptions,
resultsDir: "allure-results"
},

Contributor guide

Open the contributing guide

Research direction

Start with the reported Cucumber configuration, especially the allure-cucumberjs reporter and resultsDir set to allure-results. Reproduce the missing-test behavior locally with the listed Node, Cucumber.js, Playwright, and allure-cucumberjs versions, then compare the generated Allure results with cucumber.json and cucumber.xml. Done means all executed test cases appear in the Allure report.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, node.js, playwright, typescript
Domain
ci-cd, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.