pytest-dev / pytest-dev/pytest-timeout

Report timeouts with a TIMEOUT status instead of FAILED

Open
#87 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
258
Forks
69
No merged PRs in 30d

Description

I am the author of PR qtile/qtile#2331 and want to add pytest-timeout to the qtile test suite.
The change I made to the conftest.py was to implement a TIMEOUT status plus inline timings to make it more obvious which tests fail and which ones time out. Two of the maintainers commented that they'd rather prefer it to have this functionality built into pytest-timeout and I agreed. So my shot at it is here and we'd love to get some feedback if something like that could be added to pytest-timeout.

Example output, luckily without `TIMEOUT` or `FAILED (...)`: 
  test/test_bar.py::test_basic PASSED (0.001)                              [  2%]
  test/test_bar.py::test_basic PASSED (0.116)                              [  2%]
  test/test_bar.py::test_basic PASSED (0.036)                              [  2%]
  test/test_bar.py::test_singlespacer PASSED (0.001)                       [  2%]
  test/test_bar.py::test_singlespacer PASSED (0.114)                       [  2%]
  test/test_bar.py::test_singlespacer PASSED (0.035)                       [  2%]
  test/test_bar.py::test_nospacer PASSED (0.001)                           [  2%]
  test/test_bar.py::test_nospacer PASSED (0.115)                           [  2%]
  test/test_bar.py::test_nospacer PASSED (0.034)                           [  2%]
  test/test_bar.py::test_configure_broken_widgets PASSED (0.001)           [  2%]
  test/test_bar.py::test_configure_broken_widgets PASSED (0.112)           [  2%]
  test/test_bar.py::test_configure_broken_widgets PASSED (0.039)           [  2%]
  test/test_bar.py::test_bar_hide_show_with_margin PASSED (0.001)          [  2%]
  test/test_bar.py::test_bar_hide_show_with_margin PASSED (0.207)          [  2%]
  test/test_bar.py::test_bar_hide_show_with_margin PASSED (0.035)          [  2%]
  test/test_check.py::test_check_default_config SKIPPED (needs mypy)       [  3%]
  test/test_check.py::test_check_default_config PASSED (0.000)             [  3%]
  test/test_check.py::test_check_bad_syntax SKIPPED (needs mypy)           [  3%]
  test/test_check.py::test_check_bad_syntax PASSED (0.000)                 [  3%]

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the referenced qtile/test/conftest.py implementation at lines 64-96 and compare its TIMEOUT status and inline timings with pytest-timeout's behavior. Trace the relevant pytest-timeout entry points and determine what tests would demonstrate that timed-out tests are distinguished from failures and timings are reported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.