pytest-dev / pytest-dev/pytest
Presence of a plugin causes doctests a shared namespace to fail traceback with assertion
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
In jaraco/jaraco.test#3, I've described an issue that appears to implicate some of the most edgy features of pytest and Python, in particular:
- assert rewrite
- doctests
- namespace packages
- plugins
Oh, joy.
In summary, the issue seems to be that:
The presence of an empty module as a plugin in a namespace shared by the package under test causes assert exceptions in doctests to be re-written, causing the output from those doctests to fail where they wouldn't otherwise, even if the plugin is disabled with -p no:....
As you can see from the tox execution in the downstream report, I'm using pytest 6.1.2 on Python 3.9.0 with jaraco.test 4.0.1 (the implicated plugin) on macOS 11.0.1 (latest stable everything as of this writing).
I don't yet have a minimal example, though you can check out jaraco/jaraco.itertools@e9f23a4 and run tox -- -k assert to see the error.
I expect (though haven't verified) that any of the following remedies might be possible:
- Move the plugin to a package not sharing the
jaraconamespace. I was considering creatingpytest-enableranyway. - Replace the bare
assertwith araise AssertionError. I'll test that soon. - Update the test expectation to expect the rewritten assert output.
- Replace the doctest with a unit test.
None of these workarounds would address the root cause, but instead avoid the confounding factors leading to the failed expectation.
I recognize this issue is probably too obscure to expect it to be fixed upstream, but I could really use some advice on how to proceed with an investigation. In particular, can you explain or point me to the implementation where doctests get special treatment for assert-rewrite (or vice-versa), especially any code that might be relevant for selectively affecting the modules-under-test or plugins?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by checking out jaraco/jaraco.itertools at e9f23a4 and run tox -- -k assert to reproduce the doctest failure. Then trace pytest's doctest handling and assertion rewriting, focusing on the effect of an empty plugin module in the shared jaraco namespace and whether disabling the plugin changes it. Done means documenting a minimal reproduction and a clear root cause or upstream direction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100