pytest-dev / pytest-dev/pytest
Have an option to implement new test result / outcome
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
I've encountered a situation where i need to defer 2 types of skip statuses (in addition to xfail) and perform different actions + reporting at the end of the test.
Suggestion:
Have an option to create new test status outcome (e.g. pytest.feature_not_ready), that inherent from pytest OutcomeException class and allows us to increase the diversity of test results.
For example:
@pytest.fixture
def check_precondition(precondition):
if not precondition:
do_action_a()
pytest.skip(f"precondition {precondition} not met, can't continue the test")
@pytest.fixture
def check_exisiting_bugs(bugs_list, metafunc):
test_name = metafunc.definition.name
for bug in bugs_list:
if bug.test_name == test_name:
do_action_b()
pytest.feature_not_ready(f'will not run test {test_name} with opened bug {bug}')
I've tried to use the pytest_exception_interact and create a custom exception to be raised for the new test outcome, using this exception control/add my actions.
The problem with this solution is that the exception is caught after test result is determined → test is failed and i can't control the result (e.g. skip #2).
If at the end of the exception class, the pytest.skip will be raised - the exception itself won't be caught by pytest_exception_interact and it will be identical to regular skip test result (which i try to avoid and add another)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the pytest_exception_interact hook and OutcomeException handling described in the issue, then compare them with pytest.skip behavior. The work is done when a distinct, user-defined outcome can control the test result and end-of-test reporting without being treated as a regular skip.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100