LogCaptureHandler.reset() does not reset log level - allows mutation between tests. Should it?

• a detailed description of the bug or problem you are having

This commit which begins reusing LogCaptureHandler instead of creating it each test allows for side-effects to persist in between tests.

I discovered this when LogCaptureHandler_instance.setLevel() broke downstream tests because the level had been set above the default (NOTSET AKA 0).

I've updated my code but wonder - should LogCaptureHandler.reset() also reset the log level to NOTSET as per pytest < 6.0.0 behavior?

• output of pip list from the virtual environment you are using

Package            Version
------------------ -------
attrs              20.2.0
importlib-metadata 2.0.0
iniconfig          1.1.1
packaging          20.4
pip                20.1.1
pluggy             0.13.1
py                 1.9.0
pyparsing          2.4.7
pytest             6.1.1
setuptools         49.2.0
six                1.15.0
toml               0.10.1
wheel              0.34.2
zipp               3.3.1

• pytest and operating system versions

pytest 6.1.1, Ubuntu 18.04.5 LTs

• minimal example if possible

# test_reset.py
import logging


def log_a_warning():
    logging.warning("this is a message")


def test_caplog(caplog):
    "Test that caplog works."
    log_a_warning()
    assert len(caplog.messages) == 1


def test_mutate_handlers():
    """
    Mutates handlers.

    Begs the question: should LogCaptureHandler.reset() also reset the loglevel?
    """
    logger = logging.getLogger()
    for handler in logger.handlers:
        handler.setLevel(logging.CRITICAL)


def test_caplog_again(caplog):
    """
    Identical to test_caplog, but now the test fails.

    This is because I mutated the handler and LogCaptureHandler.reset() does not reset the log level.
    """
    log_a_warning()
    assert len(caplog.messages) == 1

with output

$ pytest
====================================================== test session starts ======================================================
platform linux -- Python 3.7.8, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/miker985/tmp/pytest-loghandler-reset-issue
collected 3 items

test_reset.py ..F                                                                                                         [100%]

=========================================================== FAILURES ============================================================
_______________________________________________________ test_caplog_again _______________________________________________________

caplog = <_pytest.logging.LogCaptureFixture object at 0x7f922c7fdb90>

    def test_caplog_again(caplog):
        """
        Identical to test_caplog, but now the test fails.

        This is because I mutated the handler and LogCaptureHandler.reset() does not reset the log level.
        """
        log_a_warning()
>       assert len(caplog.messages) == 1
E       assert 0 == 1
E        +  where 0 = len([])
E        +    where [] = <_pytest.logging.LogCaptureFixture object at 0x7f922c7fdb90>.messages

test_reset.py:32: AssertionError
==================================================== short test summary info ====================================================
FAILED test_reset.py::test_caplog_again - assert 0 == 1
================================================== 1 failed, 2 passed in 0.02s ==================================================

Start with LogCaptureHandler.reset() and run the reproducer in test_reset.py to observe how a handler level changed by one test affects the next. Compare the reset behavior with the reported pytest versions and determine whether completion should include restoring the level to NOTSET, with the reproducer passing afterward.