pytest-dev / pytest-dev/pytest

"ERROR at" test case name headings ambiguous if they use the same test case function name

Open
#14,902 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

First of all, thanks for providing and maintaining pytest!

Given a minimal setup like this:

# conftest.py
import pytest

@pytest.fixture
def fixture():
    raise Exception()
# test_1.py and test_2.py
def test(fixture):
    pass

The "ERROR at" section being printed is ambiguous. You can't tell which section is referring to which test case. Both say ERROR at setup of test.

================================ ERRORS =================================
________________________ ERROR at setup of test _________________________

    @pytest.fixture
    def fixture():
>       raise Exception()
E       Exception

conftest.py:5: Exception
________________________ ERROR at setup of test _________________________

    @pytest.fixture
    def fixture():
>       raise Exception()
E       Exception

conftest.py:5: Exception

Given the short test summary section, sometimes it's possible to "reverse engineer" which section is referring to which test case especially if they were caused by different exception types and/or lines of code causing the issue. It requires scrolling back and forth, tough.

The "FAILURES" section uses the same headings. At least there's 1 line of code within the stack trace revealing the originally affected test case usually, though.

Afaik there is neither a necessity nor a recommendation to make test case function names unique. So I wonder, why it's not printing its full name by default for these sections.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the minimal conftest.py and duplicate test_1.py/test_2.py setup, then locate the code that formats the ERROR and FAILURE section headings. Update the headings so identical test function names are distinguishable by their full test identity, and verify the output no longer ambiguously labels both cases as "ERROR at setup of test".

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.