apache / apache/buildstream

Make hanging pytest tests more debuggable

Open
#420 2 comments 0 reactions 0 assignees View on GitHub
enhancement infrastructure tests
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/420)
In GitLab by [[Gitlab user @tlater]](https://gitlab.com/tlater) on Jun 11, 2018, 18:29

## Background

[//]: # (Provide a background or the root/source that justifies this task or action.)

One of the more frequent headaches we've been running into are test cases that fail to fail because a subprocess ends up raising an exception in a location where it isn't safe to do so yet - usually those cases are impossible to actually reach in production, but during development they aren't uncommon.

This tends to result in a pytest run that just continues forever, and requires some non-obvious hackery to debug.

## Task description

[//]: # (Short summary of the action to be executed)

One way to resolve this would be to add timeouts of some sort to our test cases, which if they expire force terminate buildstream, printing a nice stack trace. This would make the actual error visible, and save a lot of time spent fighting the test suite.

[pytest-timeout](https://pypi.org/project/pytest-timeout/) might be just what we need.

Obviously finding a reasonable timeout time isn't simple, and I would dislike actually setting timeouts for individual tests, but having a default argument wth a reasonably high timeout value could make running `./setup.py test` much more useful.

This is just one suggestion, of course, feel free to discuss :)

## Acceptance Criteria

[//]: # (Acceptance criteria should follow the S.M.A.R.T. principle https://en.wikipedia.org/wiki/SMART_criteria )

BuildStream should detect and print stack traces on "hangs" - i.e., test cases that fail to continue because a subprocess is being waited for indefinitely.

Contributor guide

Open the contributing guide

Research direction

Start with the ./setup.py test entry point and review the pytest-timeout suggestion. Define a default timeout that detects indefinitely waiting tests, prints stack traces, and force-terminates BuildStream; done means the acceptance criterion is met without requiring per-test timeout configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.