pytest-dev / pytest-dev/pytest

setUpClass errors, but setUp fails on unittest.TestCase

Open
#11,604 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

plugin: unittest
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

When a fixture fails, Pytest reports an error:

import pytest


@pytest.fixture
def fail():
    assert False


def test_pass(fail):
    pass
============================= test session starts ==============================
platform linux -- Python 3.9.18, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/tmp.CAhPLzhZ0P
collected 1 item

test_pytest.py E                                                         [100%]

==================================== ERRORS ====================================
_________________________ ERROR at setup of test_pass __________________________

    @pytest.fixture
    def fail():
>       assert False
E       assert False

test_pytest.py:6: AssertionError
=========================== short test summary info ============================
ERROR test_pytest.py::test_pass - assert False
=============================== 1 error in 0.03s ===============================

The same thing happens when setUpClass fails on a unittest.TestCase:

import unittest


class Test(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        assert False

    def test_sscce(self):
        assert True
============================= test session starts ==============================
platform linux -- Python 3.9.18, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/tmp.CAhPLzhZ0P
collected 1 item

test_unittest.py E                                                       [100%]

==================================== ERRORS ====================================
______________________ ERROR at setup of Test.test_sscce _______________________

cls = <class 'test_unittest.Test'>

    @classmethod
    def setUpClass(cls):
>       assert False
E       assert False

test_unittest.py:7: AssertionError
=========================== short test summary info ============================
ERROR test_unittest.py::Test::test_sscce - assert False
=============================== 1 error in 0.04s ===============================

However, if setUp fails, Pytest reports a failure:

import unittest


class Test(unittest.TestCase):
    def setUp(self):
        assert False

    def test_sscce(self):
        assert True
============================= test session starts ==============================
platform linux -- Python 3.9.18, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/tmp.CAhPLzhZ0P
collected 1 item

test_unittest.py F                                                       [100%]

=================================== FAILURES ===================================
_______________________________ Test.test_sscce ________________________________

self = <test_unittest.Test testMethod=test_sscce>

    def setUp(self):
>       assert False
E       assert False

test_unittest.py:6: AssertionError
=========================== short test summary info ============================
FAILED test_unittest.py::Test::test_sscce - assert False
============================== 1 failed in 0.03s ===============================

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the three snippets to compare how fixture, setUpClass, and setUp errors are reported. Then trace pytest's unittest integration and reporting path, and add regression coverage once the intended error classification is confirmed; done means the agreed behavior is captured by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.