pytest-dev / pytest-dev/pytest

Disable pytest logging plugin for single test

Open
#9,018 7 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

plugin: capture plugin: logging
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

What's the problem this feature will solve?

I'm trying to test a CLI tool which produces stdout/stderr output through logging.
I need to make sure the logging is properly configured and stdout/stderr is indeed produced with proper formatting.
I don't want to check configuration, but actual output of the command while retaining ability to mock things (this is why running cli tool in subprocess call is not an option).

Describe the solution you'd like

Any solution which won't easily break with pytest upgrade and prevent pytest from messing with logging configuration will be good.
Ideally pytest would also be able to restore the logging configuration before test was started, but that seems like a another feature.

Solution ideas:

def suspend_logcapture(pytestconfig):
    manager = pytestconfig.pluginmanager
    plugin = manager.get_plugin("logging-plugin")
    manager.unregister(plugin)
    yield
    manager.register(plugin)  # currently breaks on pytest_addoption
Alternative Solutions

right now I'm using following which I consider bad since I rely on pytest internals:

@pytest.fixture
def suspend_logcapture(pytestconfig):
    from _pytest.logging import LogCaptureHandler
    @contextlib.contextmanager
    def context(*args, **kwargs):
        yield LogCaptureHandler()

    with patch("_pytest.logging.catching_logs", context):
        yield

Tried also going through temporary removing handlers & settings set by pytest

@pytest.fixture
def suspend_logcapture():
    logger = logging.getLogger()
    config = {
        "disabled": False,
        "handlers": [],
        "level": logging.NOTSET,
    }
    old_config = {attr: getattr(logger, attr) for attr in config}
    for attr, value in config.items():
        setattr(logger, attr, value)
    yield
    for attr, value in old_config.items():
        setattr(logger, attr, value)

but pytest thinks nothing of it, as somehow the logging plugin is given yet another chance to modify root logger before the actual test gets run and sees its handler was removed and readds it

Additional context

solution ideas are based on:
https://github.com/kvas-it/pytest-console-scripts
https://pypi.org/project/pytest-disable-plugin/

As to why I didn't use them - code audits are a thing and costs time; also the core the problem seemed to me like more and issue with pytest itself.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the _pytest.logging entry points mentioned in the issue, especially LogCaptureHandler, catching_logs, and plugin registration through pytest_addoption. Compare the proposed caplog.disabled() behavior with temporarily disabling the logging plugin, including restoration before and after a test. Done means a supported per-test way to prevent pytest from altering logging configuration without relying on pytest internals.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.