allure-framework / allure-framework/allure2

False positive test count when same test is associated with multiple features.

Open
#1,885 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
5.5k
Forks
782
Avg merge
2d 33m
Merged PRs (30d)
16

Description

#### I'm submitting a
- [X] bug report

#### What is the current behavior?
When for a single test case many feature names are provided, the report counts the same test case that many times, causing wrong count of tests in behavior tab.

#### If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
```
@allure.suite("TEST SUITE -- HERE")
@allure.testcase("TEST CASE -- HERE")
def test_first_test_case(test_data_set):
allure.dynamic.feature('TEST FEATURE - A')
allure.dynamic.feature('TEST FEATURE - B')
allure.dynamic.feature('TEST FEATURE - C')
```

#### What is the expected behavior?
Same test will be available under - "TEST FEATURE - A", "TEST FEATURE - B" & "TEST FEATURE - C", but the test count should be 1.

#### What is the motivation / use case for changing the behavior?
This makes it difficult to prepare test metric by parsing data from allure report. Also, it present a wrong count to the user.

#### Please tell us about your environment:

- Allure version: 2.10.0
- Test framework: pytest@6.2.2
- Allure adaptor: 'allure-pytest': '2.8.34'

![image](https://user-images.githubusercontent.com/84368829/123115404-b5ad1b80-d45d-11eb-8c27-a386e612dda7.png)
![image](https://user-images.githubusercontent.com/84368829/123115640-decdac00-d45d-11eb-9f50-67cb61d80642.png)

Contributor guide

Open the contributing guide

Research direction

Reproduce the report with the provided pytest example using three dynamic feature names, then inspect how the behavior tab aggregates test results. The fix is complete when the test appears under all three features while the displayed test count remains 1; add or update coverage for this case if the project’s existing report tests identify a suitable location.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.