pytest-dev / pytest-dev/pytest-html

pytest-html version 3.1.1 bug- Internal error in pytest_runtest_makereport causes test to show as Passed in HTML report

Open
#1,027 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
779
Forks
260
PR merge metrics
No merged PRs in 30d

Description

Description

When an internal error occurs inside the pytest_runtest_makereport hook,
pytest-html 3.1.1 does not receive a valid report outcome and silently
defaults to displaying the test as Passed in the HTML report.

Environment

  • pytest-html: 3.1.1
  • pytest: 7.1.2 or 9.0.3
  • Python: 3.13.7
  • OS: Windows

Expected Behavior

Test should be marked as ERROR in the HTML report.

Actual Behavior

Test is marked as Passed in the HTML report, giving a false green result.

Steps to Reproduce

  1. Trigger an exception inside pytest_runtest_makereport hook for eg raise RuntimeError("BOOM")
  2. Run pytest with --html=report.html
  3. Open the HTML report
  4. Observe the affected test shows as Passed

Additional Notes

pytest-html 4.2.0 handles this differently — the report is empty instead
of showing a false Passed — but neither behavior is correct.
the correct expected outcome is debatable:
ERROR — represents a framework/fixture level problem, not a test assertion failure. This is probably the most accurate.
FAILED — could argue this since something clearly went wrong.
At minimum — not Passed, since that's factually wrong.
-In case of keyboard interrupt too same issue

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with pytest-html 3.1.1, pytest, --html=report.html, and an exception or keyboard interrupt in the pytest_runtest_makereport hook. Trace how the hook result reaches the HTML report and ensure the affected test is not displayed as Passed; the intended ERROR versus FAILED outcome still needs resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
html, python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.