pytest-dev / pytest-dev/pytest
Assertion rewriting in doctests
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
As a feature request, it would be convenient if doctests did the same assertion rewriting as regular tests. For example, here is the same failing test with and without doctests:
a = False
b = True
def bar():
"""
>>> assert a == b
"""
pass
def test_bar():
assert a == b
Result of running with pytest version 5.2.1:
$ pytest --doctest-modules
====================================== test session starts =======================================
platform darwin -- Python 3.6.3, pytest-5.2.1, py-1.5.2, pluggy-0.13.0
rootdir: .../pytest_test
collected 2 items
test_foo.py FF [100%]
============================================ FAILURES ============================================
_____________________________________ [doctest] test_foo.bar _____________________________________
005
006 >>> assert a == b
UNEXPECTED EXCEPTION: AssertionError()
Traceback (most recent call last):
File ".../python3.6/doctest.py", line 1330, in __run
compileflags, 1), test.globs)
File "<doctest test_foo.bar[0]>", line 1, in <module>
AssertionError
.../pytest_test/test_foo.py:6: UnexpectedException
____________________________________________ test_bar ____________________________________________
def test_bar():
> assert a == b
E assert False == True
test_foo.py:11: AssertionError
======================================= 2 failed in 0.07s ========================================
The regular test has assert rewriting that makes the problem easy to diagnose, while the doctest requires the user to go back and add debugging statements (which themselves are a little tricky to get right in doctests).
Apologies if this has been discussed before -- I guessed it would have been, but couldn't find a an issue about it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading pytest's doctest integration and running the reported pytest --doctest-modules example alongside the regular test. Trace how doctest assertions are collected and reported; done means failing doctest assertions provide useful rewritten diagnostics comparable to regular pytest assertions, with relevant coverage added.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100