allure-framework / allure-framework/allure-python

allure-pytest: using the same @pytest.mark.parametrize for tests with different browsers appear as retries of the same test in the test report.

Aperta
#752 7 commenti 1 reazione 0 assegnatari Vedi su GitHub
bug contribute theme:pytest
Lingua principale
Python
Stelle
814
Fork
260
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Hi, I am conducting login tests on different browsers using `pytest-playwright`. I am using `@pytest.mark.parametrize` to run tests with different emails and passwords. However, I noticed that in the allure report, my tests for Edge and Chrome are grouped together as the same test case. The test that runs first becomes the retries for the subsequent test case. I want them to appear as separate tests.

What I would want it to be like would be:
```
test_file_name
|__ Chrome
| |__ Test if the website can be successfully logged in
|
|__ Edge
| |__ Test if the website can be successfully logged in
|
|__ Firefox
| |__ Test if the website can be successfully logged in
```
I've tried using `allure.dynamic.tag` and `allure.dynamic.parameter`, but they didn't help. Thanks in advance.

#### I'm submitting a ...
- [ ] bug report
#### What is the current behavior?
![image](https://github.com/allure-framework/allure-python/assets/56353753/6138f0a8-4073-47d2-9bcd-d92644a597b7)

#### If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
test_login.py
```
login_correct=[{"email": "123", "password": "123"}]

@allure.story("Test if the website can be successfully logged in")
@allure.title("{test_input}")
@allure.description("Enter the website and input correct email and password to check if can login.")
@pytest.mark.parametrize("test_input",login_correct)
def test_login_success(setup,test_input):
page = setup
login(page, test_input["email"], test_input["password"])
expect(page.get_by_text("Dashboard").first).to_be_visible(timeout=50*1000)
```

conftest.py
```
@pytest.fixture(scope="function")
def setup(page: Page,pytestconfig):
page.set_viewport_size({"width": 1920, "height": 1080})
page.goto(".....")

browser = pytestconfig.getoption("--browser")
browser_channel = pytestconfig.getoption("--browser-channel")
if browser_channel != None:
allure.dynamic.feature(f"{browser_channel}")
allure.dynamic.tag(browser_channel)
allure.dynamic.parameter("browser_name",browser_channel)
else:
allure.dynamic.feature(f"{browser[0]}")
allure.dynamic.tag(browser[0])
allure.dynamic.parameter("browser_name",browser[0])
yield page
```
I am testing different browsers using the following command:
```
pytest -n auto --browser-channel=chrome --alluredir=allure-results
pytest -n auto --browser=firefox --alluredir=allure-results
pytest -n auto --browser-channel=msedge --alluredir=allure-results
```

#### What is the expected behavior?
The Edge test case and Chrome Test case need to be seperate.

#### What is the motivation / use case for changing the behavior?

#### Please tell us about your environment:

- Allure version: 2.22.3
- Test framework: pytest@7.3.2
- Allure adaptor: allure-pytest@2.13.2
- pytest-playwright 0.3.3

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia con i file forniti test_login.py e conftest.py, quindi riproduci il report usando i tre comandi pytest per Chrome, Firefox ed Edge. Confronta il modo in cui i risultati Allure generati identificano ogni esecuzione del browser; il lavoro è completato quando il report mostra casi di test separati per ogni browser invece dei tentativi ripetuti.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
playwright, python
Ambito
testing-qa
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
45/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.