Rerun flaky tests after the others
Open
enhancement
- Dominant language
- Python
- Stars
- 398
- Forks
- 57
- PR merge metrics
- No merged PRs in 30d
Description
If a flaky test fails, it looks like it's rerun immediately.
For flaky integration tests which run against a third party, the external service may only be temporarily down. Running the same test immediately won't help much.
But deferring it, and sticking it at the end of the queue so it's run after all the remaining tests, might give the external service enough time to sort itself out and let the flaky test pass.
I'm seeing this now (using pytest). A flaky test passes in one Travis job, but fails in the next.
Is there an option for deferring flaky tests until after the rest of the test suite?
If not, would it be possible to add one?
Thank you!
Contributor guide
Assessment
This issue has not been assessed yet.