pytest-dev / pytest-dev/pytest-timeout
Set timeout for both test function only and test function + all fixture setup and teardown
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 258
- Forks
- 69
- PR merge metrics
- No merged PRs in 30d
Description
It would be nice if it was possible to set a separate timeout for calling the function instead of just setting that the timeout applies only to the function. Currently I can set timeout_func_only = true in pyproject.toml, but that means that all timeouts apply only to the test function.
Concretely, The problem is that I have two kind of timeouts.
I would like to enforce that a selection of tests are fast when fixtures that can be cached are ignored. I currently run those like this:
pytest -m "not slow" tests/ --timeout=10
with timeout_func_only = true in pyproject.toml.
This is to maintain a set (the majority in our case, more than 2k), can be ran quickly (takes less than 40s with caching and pytest-xdist). Each of those tests should take less than 0.1s so a generous timeout of 10s seems appropriate. Only setting a session timeout for this run is not as reliable as it can vary widely because of pytest-xdist.
However, when running all tests in a CI runner, I need to set an appropriate large timeout (roughly 300s) for each test in order to avoid that a hanging test racks up billable minutes. A session timeout works, but needs to be set to an even larger timeout in order to avoid spurious fails due to variation. For this use case I would like to have timeout_func_only = false as slow fixtures matters for billable minutes.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing timeout_func_only configuration and the timeout handling for test functions, fixtures, setup, and teardown. Define how separate function-only and full-test timeouts should be configured and verified for both the fast test run and the CI use case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100