Doctests in __main__ are ignored

Prior to pytest 7, pytest would exercise doctests in __main__. https://github.com/pytest-dev/pytest/pull/8949 changed that expectation (and for good reason).

In my use-case, I always put a __name__ == '__main__' check in my entry points, including __main__ and sometimes supply functions in that module that I'd like to have doctested. There are several ways I rely on doctests:

• Test explicit behaviors in functions of the module.
• Even if there are no doctests at all, doctesting a module performs syntax validation on it, ensuring that invalid syntax will be caught by the test suite, that the module is safe to import (doesn't execute on import), and that it has its dependencies met.
• A doctest can perform certain important assertions on the module, such as the presence or absence of key attributes or functions.

Most if not all of the benefits described in In defense of doctests apply to __main__ as much as any other package module.

I've been working under the assumption that __main__ modules were doctested in all of my projects and only today learned that they're not. I disagree that these modules should be special-cased just because some projects are unwilling to make them import-safe. Surely, pytest would consider it invalid if a user were to complain about any other module in the package failing to import because it has undesirable import-time behaviors, e.g.

# pkg/foo.py
raise SystemExit(1)

Yes, it's true that __main__.py has some special semantics, but so also does __init__.py, but that's not justification for excluding __init__ from doctests.

I note that there's a related issue https://github.com/pytest-dev/pytest/issues/11716, which proposes to give users more control to exclude things for doctests. If implemented, that approach could potentially satisfy the need for some projects to exclude __main__.

Even prior to addressing that issue, it would have been possible for users to exclude __main__ modules by adding it to their pytest_collect_ignore (or is it pytest_collectignore; I can never remember).

Can we reconsider supporting __main__ for doctests?

Start by reviewing the behavior change in pull request 8949 and the related issue 11716, then inspect how pytest handles main.py and pytest_collect_ignore. Done means doctests in main are supported while projects can still explicitly exclude them.