microsoft / microsoft/playwright-pytest
retain-on-failure discards trace/video if teardown/setup fails in fixtures
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 563
- Forks
- 90
- Avg merge
- 11m
- Merged PRs (30d)
- 1
Description
From what I can see, if a fixture fails in setup/teardown, the pytest --output .out --tracing retain-on-failure discards the trace. And at least sometimes the user will want the trace to be kept.
The code only looks at whether the rep_call has failed. https://github.com/microsoft/playwright-pytest/blob/main/pytest_playwright/pytest_playwright.py#L224-L226
But that ignores issues in the setup/teardown, e.g. pytests the example code
# content of conftest.py
import pytest
@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
# execute all other hooks to obtain the report object
outcome = yield
rep = outcome.get_result()
# set a report attribute for each phase of a call, which can
# be "setup", "call", "teardown"
setattr(item, "rep_" + rep.when, rep)
@pytest.fixture
def something(request):
yield
# request.node is an "item" because we use the default
# "function" scope
if request.node.rep_setup.failed:
print("setting up a test failed!", request.node.nodeid)
elif request.node.rep_setup.passed:
if request.node.rep_call.failed:
print("executing test failed", request.node.nodeid)
So if a user writes a fixture, that does something with a page in it's setup or teardown, and that setup or teardown code fails, the trace/video will be discarded. That's because at this point pytest will set rep_setup.failed (or rep_teardown) to be true, rather than rep_call.failed, I believe. Docs for the runtest_protocol
@pytest.fixture()
def logged_in(page: Page) -> Page:
"""Log in"""
log_in_flow(page)
yield page
log_out_flow(page)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pytest_playwright/pytest_playwright.py at the trace-retention logic around lines 224-226, then review pytest's runtest_protocol and the fixture report example linked in the issue. Done means traces and videos are retained when fixture setup or teardown fails, not only when rep_call.failed is true.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100