pytest-dev / pytest-dev/pytest-asyncio

Would you find it useful if you could reliably fail tests that have unawaited coroutines?

Open
#67 5 comments 21 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.7k
Forks
207
Avg merge
5h 35m
Merged PRs (30d)
9

Description

Right now, this test passes:

async def do_something_broken():
    assert False

@pytest.mark.asyncio
async def test_something_broken():
    do_something_broken()

The reason is that we forgot the await in test_something_broken, so the broken code never actually ran. Oops. Python does issue a RuntimeWarning: coroutine 'do_something_broken' was never awaited, and recent pytest will print this at the end of tests, but this has a few issues:

  • if your CI is green then how often do you click through to check for non-fatal warnings?
  • since the warning isn't issued until the coroutine is garbage collected, on PyPy this can happen in some random other test, or if the test is near the end of the run it might never happen at all. E.g. with latest pypy3, pytest, and pytest-asyncio, the above test doesn't issue any warnings at all:
============================= test session starts ==============================
platform linux -- Python 3.5.3[pypy-5.8.0-beta], pytest-3.2.2, py-1.4.34, pluggy-0.4.0
rootdir: /tmp, inifile:
plugins: cov-2.5.1, catchlog-1.2.2, asyncio-0.7.0
collected 1 item                                                                

../../tmp/test.py .

=========================== 1 passed in 0.02 seconds ===========================

I'm considering proposing a new feature for Python 3.7, that would make it so pytest-asyncio could do:

# Ask Python to start maintaining a list of unawaited coroutines
sys.set_unawaited_coroutine_tracking(True)
try:
    ... run the test ...
finally:
    # Get the unawaited coroutines
    unawaited_coroutines = sys.get_and_clear_unawaited_coroutines()
    sys.set_unawaited_coroutine_tracking(False)
    if unawaited_coroutines:
        # Issue an error that points to the actual problem
        raise RuntimeError(f"Unawaited coroutines: {unawaited_coroutines}")

(Names etc. to be bikeshedded later; this is "API 2" in https://github.com/python-trio/trio/issues/79#issuecomment-325188030)

This way you could deterministically detect unawaited coroutines, reliably attribute them to the correct test, and cause it to fail with a useful error message.

Is this an API that you'd want to take advantage of if it were available?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the unawaited-coroutine example using pytest.mark.asyncio and test_something_broken, then review the proposed sys.set_unawaited_coroutine_tracking and sys.get_and_clear_unawaited_coroutines APIs. The issue does not identify implementation files or tests, and completion depends on a new Python API whose design is still hypothetical.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.