pytest-dev / pytest-dev/pytest
"ERROR at" test case name headings ambiguous if they use the same test case function name
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
First of all, thanks for providing and maintaining pytest!
Given a minimal setup like this:
# conftest.py
import pytest
@pytest.fixture
def fixture():
raise Exception()
# test_1.py and test_2.py
def test(fixture):
pass
The "ERROR at" section being printed is ambiguous. You can't tell which section is referring to which test case. Both say ERROR at setup of test.
================================ ERRORS =================================
________________________ ERROR at setup of test _________________________
@pytest.fixture
def fixture():
> raise Exception()
E Exception
conftest.py:5: Exception
________________________ ERROR at setup of test _________________________
@pytest.fixture
def fixture():
> raise Exception()
E Exception
conftest.py:5: Exception
Given the short test summary section, sometimes it's possible to "reverse engineer" which section is referring to which test case especially if they were caused by different exception types and/or lines of code causing the issue. It requires scrolling back and forth, tough.
The "FAILURES" section uses the same headings. At least there's 1 line of code within the stack trace revealing the originally affected test case usually, though.
Afaik there is neither a necessity nor a recommendation to make test case function names unique. So I wonder, why it's not printing its full name by default for these sections.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the minimal conftest.py and duplicate test_1.py/test_2.py setup, then locate the code that formats the ERROR and FAILURE section headings. Update the headings so identical test function names are distinguishable by their full test identity, and verify the output no longer ambiguously labels both cases as "ERROR at setup of test".
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100