box / box/flaky

Flaky and pytest-check don't fail when used together

Open
#162 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
398
Forks
57
PR merge metrics
No merged PRs in 30d

Description

Hi,

I've also filed this under https://github.com/okken/pytest-check/issues/25, but since the issue arises only when using both plugins together, I'm not sure where the responsibilities lie, so flagging it with both projects to be sure.

This simple case fails as expected:

```
from pytest_check import check
import pytest

# @pytest.mark.flaky
def test_foo():
with check:
assert 11 == 10
```
But uncomment the `@pytest.mark.flaky` and the test now "passes".

The "Flaky Test Report" in the console prints ``test_foo passed 1 out of the required 1 times. Success!``, whereas the pytest final line says ``no tests ran`` instead of either the red ``1 failed`` (expected) or the green ``1 passed`` (which would be more consistent with the flaky report).

The exit code from pytest is 0 (i.e., tests passing) instead of either 1 (expected) or 5 (which would be more consistent with the ``no tests ran`` printout).

This was using python 3.7.3, pytest 4.6.5, pytest-check 0.3.5 and flaky 3.6.1.

Regards,
A.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.