pytest-dev / pytest-dev/pytest

Introduce new hook to better control display of items in terminal

Open
#5,047 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: proposal
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Recently I have added TestReport.count_towards_summary and TestReport.head_line properties as a way to control how items are shown/handled by the terminal plugin. After some feedback it is clear that I would need to customize other things as well.

The current approach of adding a property to TestReport so it can be overwritten by subclasses in plugins is sub-optimal: it requires access to the private TestReport class to declare it in the bases list of the subclass, and subclasses in general are not a good way to extend plugins functionality in plugins traditionally as they expose too much internal details.

I propose then that we add a new hook:

@hookspec(firstresult=True)
def pytest_report_displayopts(report, config) -> dict:
    """
    (Experimental)

    Returns a number of options related to the given report object. This is used
    by the terminal writer and other plugins to customize how some visual aspects 
    of tests are shown.
    
    This hook returns a dict, which may contain any key and value. Currently the 
    following keys are recognized by pytest:


    * ``category``: same as returned by ``pytest_report_teststatus``
    * ``letter``: same as returned by ``pytest_report_teststatus``
    * ``word``: same as returned by ``pytest_report_teststatus``
    * ``show-progress``: bool, should show progress at right margin or not
    * ``head-line``: the head line shown with longrepr output for this report, more commonly during
        traceback representation during failures.
    * ``count-towards-summary``: True if this report should be counted towards the totals shown at the end of the
        test session: "1 passed, 1 failure, etc".
    * ``nodeid-caption``: how to show the nodeid when in verbose mode; usually this is the
        bare nodeid, but plugins might add more information such as subtest context (with
        ``pytest-subtests``), repeat count (``pytest-repeat``), number of retries (``flaky`` plugin),
        or workers where the test was executed (``pytest-xdist``).

    In the interest of forward and backward compatibility, extra keys are ignored, so new keys can be
    added in the future and plugins which use those keys will still work in older pytest versions.
    """ 

A big advantage here is that it is easy to extend the customizability of how reports are displayed in the
future, without introducing new hooks or breaking backward compatibility. Also it will allow us to get rid of
xdist-specific code in the terminal plugin, as we will be able to delegate the decisions to the hook.

Thoughts?

cc @RonnyPfannschmidt

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the proposed pytest_report_displayopts hook and the terminal writer behavior it is intended to customize, including pytest_report_teststatus and the TestReport properties mentioned. Done means the hook is defined and used for the listed display and summary options while preserving the stated forward- and backward-compatibility behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.