pytest-dev / pytest-dev/pytest
Ability to provide a custom doctest checker
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
What's the problem this feature will solve?
I have a pre-existing test suite that uses doctests with a custom checker class that does some regex replacements to normalize filenames in doctest output, to make my test suite pass under Windows.
I'm having a hard time trying to switch to pytest as a test runner because I was unable to find a way of providing a custom doctest checker.
This is not the only project I've worked with that uses doctests and a regexp-based output normalizers to deal with platform (and sometimes Python version) differences; https://github.com/zopefoundation/ is full of these (although they don't currently have any plans of migrating to pytest).
Describe the solution you'd like
I think a workable interface would be to provide a fixture named doctest_checker:
import doctest
import pytest
class Checker(doctest.OutputChecker):
def check_output(self, want, got, optionflags):
want = re.sub(..., ..., want)
return doctest.OutputChecker.check_output(self, want, got, optionflags)
@pytest.fixture(scope='session')
def doctest_checker():
return Checker()
I've seen this pattern (overriding a named fixture) used to provide project-specific customizations for pytest test suites that use celery (https://docs.celeryq.dev/en/stable/userguide/testing.html#session-scope), but I'm not absolutely set on it. If it's unworkable, or if there's a better way, let's use that instead.
Alternative Solutions
I've tried looking at the pytest-doctestplus plugin, but I found no way of providing a custom doctest checker either.
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing pytest's doctest support and the referenced pytest-doctestplus alternative, then compare them with Python's doctest.OutputChecker API. Determine whether a fixture-based checker or another interface fits pytest's plugin model. Done means a documented, tested way to provide a custom checker for platform- or version-dependent doctest output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100