pytest-dev / pytest-dev/pytest-timeout

Set timeout for both test function only and test function + all fixture setup and teardown

Open
#190 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
258
Forks
69
PR merge metrics
No merged PRs in 30d

Description

It would be nice if it was possible to set a separate timeout for calling the function instead of just setting that the timeout applies only to the function. Currently I can set timeout_func_only = true in pyproject.toml, but that means that all timeouts apply only to the test function.

Concretely, The problem is that I have two kind of timeouts.

I would like to enforce that a selection of tests are fast when fixtures that can be cached are ignored. I currently run those like this:

pytest -m "not slow" tests/ --timeout=10

with timeout_func_only = true in pyproject.toml.

This is to maintain a set (the majority in our case, more than 2k), can be ran quickly (takes less than 40s with caching and pytest-xdist). Each of those tests should take less than 0.1s so a generous timeout of 10s seems appropriate. Only setting a session timeout for this run is not as reliable as it can vary widely because of pytest-xdist.

However, when running all tests in a CI runner, I need to set an appropriate large timeout (roughly 300s) for each test in order to avoid that a hanging test racks up billable minutes. A session timeout works, but needs to be set to an even larger timeout in order to avoid spurious fails due to variation. For this use case I would like to have timeout_func_only = false as slow fixtures matters for billable minutes.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing timeout_func_only configuration and the timeout handling for test functions, fixtures, setup, and teardown. Define how separate function-only and full-test timeouts should be configured and verified for both the fast test run and the CI use case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.