pytest-dev / pytest-dev/pytest

conditionally rewrite assertions / support assertion descriptions in `pytest_assertion_pass` and `pytest_assertrepr_compare` hooks

Open
#11,975 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: rewrite type: proposal
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

What's the problem this feature will solve?

in my plugin, i want to be able to customize or rewrite assertions based on the second argument to the assert statement:

assert False, "value was false"
Describe the solution you'd like

either of these two solutions:

  1. a hook to determine whether to rewrite an assertion:
    @hookimpl(firstresult=True) # first non-None result means don't rewrite
    def pytest_rewrite_assertion(item: Item, lineno: int, orig: str, description: object):
        ...
    
  2. add a new "description" argument to these existing hooks:
    def pytest_assertion_pass(item: Item, lineno: int, orig: str, expl: str, description: object):
        ...
    
    def pytest_assertrepr_compare(config: Config, op: str, left: object, right: object, description: object):
        ...
    
Additional context

this is mainly for the concept of "user-facing" vs "non-user-facing" assertions

user-facing assertions

a user-facing assertion is written as part of the test (ie. ensuring that the function being tested has the correct value)

def test_get_value():
    assert get_value(1) == 1
non-user-facing assertions

a non-user-facing assertion is used for things like narrowing types, typically in utility functions which have nothing to do with the test itself

def get_value(value: object):
    assert isinstance(value, int)
    ...

def test_get_value():
    assert get_value(1) == 1

my plugin is a robotframework test runner: https://github.com/DetachHead/pytest-robotframework. robotframework assertions are different to pytest assertions, since they use functions (called keywords) like should_be_equal, should_not_be_equal, etc.

we are in the process of converting an existing robotframework codebase to python using this plugin, however since the user-facing assertions were handled by those keywords, there are hundreds of assert statements in the codebase that were written as regular python non-user-facing assert statements and are not meant to be displayed to the user. we want to replace them all with pytest assertions though.

i know you can use register_assert_rewrite to control which modules have their assert statements rewritten, but in many cases there are user-facing and non-user-facing assertions in the same module.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the pytest_assertion_pass and pytest_assertrepr_compare hooks, along with register_assert_rewrite, to understand how assertion rewriting and descriptions are currently exposed. Compare the proposed rewrite-control hook with adding description arguments, and define done as supporting user-facing versus non-user-facing assertions for the plugin's Robot Framework use case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.