3 tests fail

========================================================================================== FAILURES ==========================================================================================
_________________________________________________________________________________ test_logging_fails_ignore __________________________________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0')>

    def test_logging_fails_ignore(testdir):
        """
        Test qt_log_ignore config option.
    
        :type testdir: _pytest.pytester.TmpTestdir
        """
        testdir.makeini(
            """
            [pytest]
            qt_log_level_fail = CRITICAL
            qt_log_ignore =
                WM_DESTROY.*sent
                WM_PAINT not handled
            """
        )
        testdir.makepyfile(
            """
            from pytestqt.qt_compat import qt_api
            import pytest
    
            def test1():
                qt_api.qCritical('a critical message')
            def test2():
                qt_api.qCritical('WM_DESTROY was sent')
            def test3():
                qt_api.qCritical('WM_DESTROY was sent')
                assert 0
            def test4():
                qt_api.qCritical('WM_PAINT not handled')
                qt_api.qCritical('another critical message')
            """
        )
        res = testdir.runpytest()
        lines = [
            # test1 fails because it has emitted a CRITICAL message and that message
            # does not match any regex in qt_log_ignore
            "*_ test1 _*",
            "*Failure: Qt messages with level CRITICAL or above emitted*",
            "*QtCriticalMsg: a critical message*",
            # test2 succeeds because its message matches qt_log_ignore
            # test3 fails because of an assert, but the ignored message should
            # still appear in the failure message
            "*_ test3 _*",
            "*AssertionError*",
            "*QtCriticalMsg: WM_DESTROY was sent*(IGNORED)*",
            # test4 fails because one message is ignored but the other isn't
            "*_ test4 _*",
            "*Failure: Qt messages with level CRITICAL or above emitted*",
            "*QtCriticalMsg: WM_PAINT not handled*(IGNORED)*",
            "*QtCriticalMsg: another critical message*",
            # summary
            "*3 failed, 1 passed*",
        ]
>       res.stdout.fnmatch_lines(lines)
E       Failed: nomatch: '*_ test1 _*'
E           and: '============================= test session starts =============================='
E           and: 'platform freebsd13 -- Python 3.9.13, pytest-7.1.3, pluggy-1.0.0'
E           and: 'PyQt5 5.15.6 -- Qt runtime 5.15.5 -- Qt compiled 5.15.5'
E           and: 'Using --randomly-seed=3508469783'
E           and: 'rootdir: /tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0, configfile: tox.ini'
E           and: 'plugins: qt-4.1.0, forked-1.4.0, hypothesis-6.55.0, cov-2.9.0, rerunfailures-10.1, xdist-2.5.0, randomly-3.12.0, typeguard-2.13.3'
E           and: 'collected 4 items'
E           and: ''
E           and: 'test_logging_fails_ignore.py F.FF                                        [100%]'
E           and: ''
E           and: '=================================== FAILURES ==================================='
E           and: '____________________________________ test3 _____________________________________'
E           and: ''
E           and: '    def test3():'
E           and: "        qt_api.qCritical('WM_DESTROY was sent')"
E           and: '>       assert 0'
E           and: 'E       assert 0'
E           and: ''
E           and: 'test_logging_fails_ignore.py:10: AssertionError'
E           and: '----------------------------- Captured Qt messages -----------------------------'
E           and: '/tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0/test_logging_fails_ignore.py:test3:9:'
E           and: ''
E           and: '    QtCriticalMsg: WM_DESTROY was sent (IGNORED)'
E       fnmatch: '*_ test1 _*'
E          with: '____________________________________ test1 _____________________________________'
E       fnmatch: '*Failure: Qt messages with level CRITICAL or above emitted*'
E          with: 'test_logging_fails_ignore.py:4: Failure: Qt messages with level CRITICAL or above emitted'
E       nomatch: '*QtCriticalMsg: a critical message*'
E           and: '----------------------------- Captured Qt messages -----------------------------'
E           and: '/tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0/test_logging_fails_ignore.py:test1:5:'
E           and: ''
E       fnmatch: '*QtCriticalMsg: a critical message*'
E          with: '    QtCriticalMsg: a critical message'
E       nomatch: '*_ test3 _*'
E           and: '____________________________________ test4 _____________________________________'
E           and: 'test_logging_fails_ignore.py:11: Failure: Qt messages with level CRITICAL or above emitted'
E           and: '----------------------------- Captured Qt messages -----------------------------'
E           and: '/tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0/test_logging_fails_ignore.py:test4:12:'
E           and: ''
E           and: '    QtCriticalMsg: WM_PAINT not handled (IGNORED)'
E           and: '/tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0/test_logging_fails_ignore.py:test4:13:'
E           and: ''
E           and: '    QtCriticalMsg: another critical message'
E           and: '=========================== short test summary info ============================'
E           and: 'FAILED test_logging_fails_ignore.py::test3 - assert 0'
E           and: 'FAILED test_logging_fails_ignore.py::test1'
E           and: 'FAILED test_logging_fails_ignore.py::test4'
E           and: '========================= 3 failed, 1 passed in 0.04s =========================='
E       remains unmatched: '*_ test3 _*'

/disk-samsung/freebsd-ports/devel/py-pytest-qt/work-py39/pytest-qt-4.1.0/tests/test_logging.py:339: Failed
------------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------------
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.13, pytest-7.1.3, pluggy-1.0.0
PyQt5 5.15.6 -- Qt runtime 5.15.5 -- Qt compiled 5.15.5
Using --randomly-seed=3508469783
rootdir: /tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0, configfile: tox.ini
plugins: qt-4.1.0, forked-1.4.0, hypothesis-6.55.0, cov-2.9.0, rerunfailures-10.1, xdist-2.5.0, randomly-3.12.0, typeguard-2.13.3
collected 4 items

test_logging_fails_ignore.py F.FF                                        [100%]

=================================== FAILURES ===================================
____________________________________ test3 _____________________________________

    def test3():
        qt_api.qCritical('WM_DESTROY was sent')
>       assert 0
E       assert 0

test_logging_fails_ignore.py:10: AssertionError
----------------------------- Captured Qt messages -----------------------------
/tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0/test_logging_fails_ignore.py:test3:9:

    QtCriticalMsg: WM_DESTROY was sent (IGNORED)
____________________________________ test1 _____________________________________
test_logging_fails_ignore.py:4: Failure: Qt messages with level CRITICAL or above emitted
----------------------------- Captured Qt messages -----------------------------
/tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0/test_logging_fails_ignore.py:test1:5:

    QtCriticalMsg: a critical message
____________________________________ test4 _____________________________________
test_logging_fails_ignore.py:11: Failure: Qt messages with level CRITICAL or above emitted
----------------------------- Captured Qt messages -----------------------------
/tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0/test_logging_fails_ignore.py:test4:12:

    QtCriticalMsg: WM_PAINT not handled (IGNORED)
/tmp/pytest-of-yuri/pytest-0/test_logging_fails_ignore0/test_logging_fails_ignore.py:test4:13:

    QtCriticalMsg: another critical message
=========================== short test summary info ============================
FAILED test_logging_fails_ignore.py::test3 - assert 0
FAILED test_logging_fails_ignore.py::test1
FAILED test_logging_fails_ignore.py::test4
========================= 3 failed, 1 passed in 0.04s ==========================
_________________________________________________________________________________ test_exceptions_dont_leak __________________________________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0')>

    @pytest.mark.xfail(
        condition=sys.version_info[:2] == (3, 4),
        reason="failing in Python 3.4, which is about to be dropped soon anyway",
    )
    def test_exceptions_dont_leak(testdir):
        """
        Ensure exceptions are cleared when an exception occurs and don't leak (#187).
        """
        testdir.makepyfile(
            """
            from pytestqt.qt_compat import qt_api
            import gc
            import weakref
    
            class MyWidget(qt_api.QtWidgets.QWidget):
    
                def event(self, ev):
                    called.append(1)
                    raise RuntimeError('event processed')
    
            weak_ref = None
            called = []
    
            def test_1(qapp):
                global weak_ref
                w = MyWidget()
                weak_ref = weakref.ref(w)
                qapp.postEvent(w, qt_api.QtCore.QEvent(qt_api.QtCore.QEvent.Type.User))
                qapp.processEvents()
    
            def test_2(qapp):
                assert called
                gc.collect()
                assert weak_ref() is None
        """
        )
        result = testdir.runpytest()
>       result.stdout.fnmatch_lines(["*1 failed, 1 passed*"])
E       Failed: nomatch: '*1 failed, 1 passed*'
E           and: '============================= test session starts =============================='
E           and: 'platform freebsd13 -- Python 3.9.13, pytest-7.1.3, pluggy-1.0.0'
E           and: 'PyQt5 5.15.6 -- Qt runtime 5.15.5 -- Qt compiled 5.15.5'
E           and: 'Using --randomly-seed=3508469783'
E           and: 'rootdir: /tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0'
E           and: 'plugins: qt-4.1.0, forked-1.4.0, hypothesis-6.55.0, cov-2.9.0, rerunfailures-10.1, xdist-2.5.0, randomly-3.12.0, typeguard-2.13.3'
E           and: 'collected 2 items'
E           and: ''
E           and: 'test_exceptions_dont_leak.py FF                                          [100%]'
E           and: ''
E           and: '=================================== FAILURES ==================================='
E           and: '____________________________________ test_2 ____________________________________'
E           and: ''
E           and: 'qapp = <PyQt5.QtWidgets.QApplication object at 0x8fc7e2430>'
E           and: ''
E           and: '    def test_2(qapp):'
E           and: '>       assert called'
E           and: 'E       assert []'
E           and: ''
E           and: 'test_exceptions_dont_leak.py:22: AssertionError'
E           and: '____________________________________ test_1 ____________________________________'
E           and: 'CALL ERROR: Exceptions caught in Qt event loop:'
E           and: '________________________________________________________________________________'
E           and: 'Traceback (most recent call last):'
E           and: '  File "/tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0/test_exceptions_dont_leak.py", line 9, in event'
E           and: "    raise RuntimeError('event processed')"
E           and: 'RuntimeError: event processed'
E           and: '________________________________________________________________________________'
E           and: 'Traceback (most recent call last):'
E           and: '  File "/tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0/test_exceptions_dont_leak.py", line 9, in event'
E           and: "    raise RuntimeError('event processed')"
E           and: 'RuntimeError: event processed'
E           and: '________________________________________________________________________________'
E           and: '----------------------------- Captured stderr call -----------------------------'
E           and: 'Exceptions caught in Qt event loop:'
E           and: '________________________________________________________________________________'
E           and: 'Traceback (most recent call last):'
E           and: '  File "/tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0/test_exceptions_dont_leak.py", line 9, in event'
E           and: "    raise RuntimeError('event processed')"
E           and: 'RuntimeError: event processed'
E           and: '________________________________________________________________________________'
E           and: 'Exceptions caught in Qt event loop:'
E           and: '________________________________________________________________________________'
E           and: 'Traceback (most recent call last):'
E           and: '  File "/tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0/test_exceptions_dont_leak.py", line 9, in event'
E           and: "    raise RuntimeError('event processed')"
E           and: 'RuntimeError: event processed'
E           and: '________________________________________________________________________________'
E           and: '=========================== short test summary info ============================'
E           and: 'FAILED test_exceptions_dont_leak.py::test_2 - assert []'
E           and: 'FAILED test_exceptions_dont_leak.py::test_1'
E           and: '============================== 2 failed in 0.04s ==============================='
E       remains unmatched: '*1 failed, 1 passed*'

/disk-samsung/freebsd-ports/devel/py-pytest-qt/work-py39/pytest-qt-4.1.0/tests/test_exceptions.py:381: Failed
------------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------------
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.13, pytest-7.1.3, pluggy-1.0.0
PyQt5 5.15.6 -- Qt runtime 5.15.5 -- Qt compiled 5.15.5
Using --randomly-seed=3508469783
rootdir: /tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0
plugins: qt-4.1.0, forked-1.4.0, hypothesis-6.55.0, cov-2.9.0, rerunfailures-10.1, xdist-2.5.0, randomly-3.12.0, typeguard-2.13.3
collected 2 items

test_exceptions_dont_leak.py FF                                          [100%]

=================================== FAILURES ===================================
____________________________________ test_2 ____________________________________

qapp = <PyQt5.QtWidgets.QApplication object at 0x8fc7e2430>

    def test_2(qapp):
>       assert called
E       assert []

test_exceptions_dont_leak.py:22: AssertionError
____________________________________ test_1 ____________________________________
CALL ERROR: Exceptions caught in Qt event loop:
________________________________________________________________________________
Traceback (most recent call last):
  File "/tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0/test_exceptions_dont_leak.py", line 9, in event
    raise RuntimeError('event processed')
RuntimeError: event processed
________________________________________________________________________________
Traceback (most recent call last):
  File "/tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0/test_exceptions_dont_leak.py", line 9, in event
    raise RuntimeError('event processed')
RuntimeError: event processed
________________________________________________________________________________
----------------------------- Captured stderr call -----------------------------
Exceptions caught in Qt event loop:
________________________________________________________________________________
Traceback (most recent call last):
  File "/tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0/test_exceptions_dont_leak.py", line 9, in event
    raise RuntimeError('event processed')
RuntimeError: event processed
________________________________________________________________________________
Exceptions caught in Qt event loop:
________________________________________________________________________________
Traceback (most recent call last):
  File "/tmp/pytest-of-yuri/pytest-0/test_exceptions_dont_leak0/test_exceptions_dont_leak.py", line 9, in event
    raise RuntimeError('event processed')
RuntimeError: event processed
________________________________________________________________________________
=========================== short test summary info ============================
FAILED test_exceptions_dont_leak.py::test_2 - assert []
FAILED test_exceptions_dont_leak.py::test_1
============================== 2 failed in 0.04s ===============================
_______________________________________________________________________________ test_qt_api_ini_config[pyqt5] ________________________________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-yuri/pytest-0/test_qt_api_ini_config3')>, monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x902e88520>, option_api = 'pyqt5'

    @pytest.mark.parametrize("option_api", ["pyqt5", "pyqt6", "pyside2", "pyside6"])
    def test_qt_api_ini_config(testdir, monkeypatch, option_api):
        """
        Test qt_api ini option handling.
        """
        from pytestqt.qt_compat import qt_api
    
        monkeypatch.delenv("PYTEST_QT_API", raising=False)
    
        testdir.makeini(
            """
            [pytest]
            qt_api={option_api}
        """.format(
                option_api=option_api
            )
        )
    
        testdir.makepyfile(
            """
            import pytest
    
            def test_foo(qtbot):
                pass
        """
        )
    
        result = testdir.runpytest_subprocess()
        if qt_api.pytest_qt_api == option_api:
            result.stdout.fnmatch_lines(["* 1 passed in *"])
        else:
            try:
                ModuleNotFoundError
            except NameError:
                # Python < 3.6
                result.stderr.fnmatch_lines(["*ImportError:*"])
            else:
                # Python >= 3.6
>               result.stderr.fnmatch_lines(["*ModuleNotFoundError:*"])
E               Failed: remains unmatched: '*ModuleNotFoundError:*'

/disk-samsung/freebsd-ports/devel/py-pytest-qt/work-py39/pytest-qt-4.1.0/tests/test_basics.py:480: Failed
------------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------------
running: /usr/local/bin/python3.9 -mpytest --basetemp=/tmp/pytest-of-yuri/pytest-0/test_qt_api_ini_config3/runpytest-0
     in: /tmp/pytest-of-yuri/pytest-0/test_qt_api_ini_config3
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.13, pytest-7.1.3, pluggy-1.0.0
PyQt5 5.15.6 -- Qt runtime 5.15.5 -- Qt compiled 5.15.5
Using --randomly-seed=655155166
rootdir: /tmp/pytest-of-yuri/pytest-0/test_qt_api_ini_config3, configfile: tox.ini
plugins: qt-4.1.0, forked-1.4.0, hypothesis-6.55.0, cov-2.9.0, rerunfailures-10.1, xdist-2.5.0, randomly-3.12.0, typeguard-2.13.3
collected 1 item

test_qt_api_ini_config.py .                                              [100%]

============================== 1 passed in 0.08s ===============================
================================================================================== short test summary info ===================================================================================
SKIPPED [1] tests/test_wait_signal.py:922: test only makes sense for PySide2, whose signals don't contain a name!
SKIPPED [1] tests/test_wait_signal.py:1206: test only makes sense for PySide, whose signals don't contain a name!
========================================================================= 3 failed, 371 passed, 2 skipped in 55.32s ==========================================================================
*** Error code 1

Version: 4.1.0
Python-3.9
FreeBSD 13.1 STABLE

Start with tests/test_logging.py, especially test_logging_fails_ignore, and reproduce the reported failures. Then run test_exceptions_dont_leak and compare its event-loop exception and ordering failures. Done means the reported tests pass with their expected summaries, without weakening their assertions.