pytest-dev / pytest-dev/pytest

Custom markers don't show custom color in 'short test summary info' section

Open
#9,961 16 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: marks topic: reporting
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Discussed in https://github.com/pytest-dev/pytest/discussions/9958

Originally posted by jeffwright13 May 14, 2022
At the end of a pytest run, you have a section called "short test summary info," which terminates in a single line that shows final metrics, all color-coded according to outcome. It doesn't appear as if this final line uses the custom color to color-code tests that are marked with custom markers. I do see that the initial "test session starts" section does color-code those tests, however.

Am I missing something or is this a deficiency in the final section processing code? I'd like to see that final line show my 'snowflake' tests in cyan!

Output:
Screen Shot 2022-05-14 at 3 07 36 PM

pytest.ini:

[pytest]
markers =
    snowflake: mark a test as flaky (may pass, may fail)

plugin.py:

def pytest_report_teststatus(report: TestReport, config: Config):
    """Put snowflake tests into a separate group from other failures."""
    if report.when == "call" and 'snowflake' in report.keywords.keys() and report.keywords["snowflake"]:
        return 'snowflake', '❄', ('SNOWFLAKE', {"cyan": True})

test1.py:

def test_aa_ok():
    print("This test doesn't have much to say, but it passes - so OK!!")
    pass

test2.py:

def test_aa_ok():
    print("This test doesn't have much to say, and it fails - not OK!!")
    assert 0

test_snowflake.py:

@pytest.mark.snowflake
def test_snowflake_1():
    assert True


@pytest.mark.snowflake
def test_snowflake_2():
    assert False

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the report using the supplied pytest.ini, plugin.py, test1.py, test2.py, and test_snowflake.py examples. Start by comparing marker coloring in the test-session output with the short test summary info section. Done means custom markers retain their configured cyan color in the final summary line, with coverage for both passing and failing snowflake tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.