pytest-dev / pytest-dev/pytest

add append option to LoggingPlugin.set_log_path and return old path

Open
#9,687 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

plugin: logging type: proposal
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

What's the problem this feature will solve?

set_log_path() currently overwrites existing log-files.
I would like to add fixtures that produce e.g. a logfile per module (scope = 'module') or for selected tests (scope = 'function').
Using set_log_path() is great for this, however the old path (if it existed) cannot be restored when upon fixture teardown. This does not allow the fixture to be e.g. combined with the --log-file commandline option for tests not using the fixture.

Describe the solution you'd like

Thus I propose to return the old path and to add an option to append to the file instead of overwriting it.

Alternative Solutions

I have tried different things, however using the set_log_path() seems the most simple way to achieve this.

Additional context

Here are the proposed changes to set_log_path:

    def set_log_path(self, fname: str, append=False) -> None:
        """Set the filename parameter for Logging.FileHandler().

        Creates parent directory if it does not exist.

        .. warning::
            This is an experimental API.
        """
        fpath = Path(fname)

        if not fpath.is_absolute():
            fpath = self._config.rootpath / fpath

        if not fpath.parent.exists():
            fpath.parent.mkdir(exist_ok=True, parents=True)

        stream = fpath.open(mode="w" if not append else "a", encoding="UTF-8")
        if sys.version_info >= (3, 7):
            old_stream = self.log_file_handler.setStream(stream)
        else:
            old_stream = self.log_file_handler.stream
            self.log_file_handler.acquire()
            try:
                self.log_file_handler.flush()
                self.log_file_handler.stream = stream
            finally:
                self.log_file_handler.release()
        old_path = None
        if old_stream:
            old_path = os.path.abspath(old_stream.name)
            old_stream.close()
        return old_path

Example of application for a fixture generating one log file per test:

It allows to write a specific logfile for selected tests and at the same time still allows the usage of --log-file from the commandline to work on all other tests. (attention, I added a pytestconfig.option.logdir option)

@pytest.fixture(scope='function')
def logfile_per_test(request, pytestconfig):
    """
    A fixture that creates a logfile per test/ function call.
    If the function is parametrized, one logfile per call(!) is created.
    """
    logging_plugin=request.config.pluginmanager.get_plugin("logging-plugin")
    filename=os.path.join(pytestconfig.option.logdir,
                          request.module.__name__ + '__' + request.node.name.replace('[','__').replace(']', '__') +".log")
    oldfile = logging_plugin.set_log_path(filename, append=False) # this overwrites the logfile!
    # write some intitial info to the logfile
    log = logging.getLogger(request.module.__name__)
    log.info(f"========= {request.module.__name__}.{request.node.name} =======================")
    
    yield
    
    # write some closing info to the logfile
    log.info(f"========= finished {request.module.__name__}.{request.node.name} ==============")
    
    # restore the old logfile, in case there had been one, and append to it
    if oldfile:
        logging_plugin.set_log_path(oldfile, append=True) # this further appends to the existing logfile!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the LoggingPlugin.set_log_path entry point and inspect its current file-handler and stream handling. Use the proposed append option and returned old path as acceptance criteria, including restoring an existing log file; run the relevant logging tests after locating them.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.