pytest-timeout doesn't work with pytest-rerunfailures.

Original report by Anonymous.

pytest-timeout plugin doesn't work when pytest-rerunfailures is used. Please see a example at the end of this bug report

• pip list of the virtual environment you are using

alabaster (0.7.10)
apipkg (1.4)
appnope (0.1.0)
astroid (1.4.9)
Babel (2.4.0)
bleach (1.5.0)
CommonMark (0.5.4)
coverage (4.3.4)
cycler (0.10.0)
decorator (4.0.11)
docopt (0.6.2)
docutils (0.13.1)
entrypoints (0.2.2)
execnet (1.4.1)
flake8 (3.3.0)
html5lib (0.999)
imagesize (0.7.1)
ipykernel (4.6.1)
ipyparallel (6.0.2)
ipython (6.0.0)
ipython-genutils (0.2.0)
ipywidgets (6.0.0)
isort (4.2.5)
jedi (0.10.2)
Jinja2 (2.9.6)
joblib (0.11)
jsonschema (2.6.0)
jupyter (1.0.0)
jupyter-client (5.0.1)
jupyter-console (5.1.0)
jupyter-core (4.3.0)
lazy-object-proxy (1.2.2)
MarkupSafe (0.23)
matplotlib (2.0.2)
mccabe (0.6.1)
mistune (0.7.4)
nbconvert (5.1.1)
nbformat (4.3.0)
nose (1.3.7)
notebook (5.0.0)
numpy (1.12.1)
pandas (0.20.1)
pandocfilters (1.4.1)
pexpect (4.2.1)
pickleshare (0.7.4)
pip (9.0.1)
prompt-toolkit (1.0.14)
psycopg2 (2.7.1)
ptyprocess (0.5.1)
py (1.4.33)
pyarrow (0.2.0)
pycodestyle (2.3.1)
pyflakes (1.5.0)
Pygments (2.2.0)
pylint (1.6.4)
pyparsing (2.1.4)
pytest (3.0.7)
pytest-cache (1.0)
pytest-instafail (0.3.0)
pytest-rerunfailures (2.1.0)
pytest-timeout (1.0.0)
python-dateutil (2.6.0)
pytz (2017.2)
pyzmq (16.0.2)
qtconsole (4.3.0)
recommonmark (0.4.0)
requests (2.14.2)
scikit-learn (0.18.1)
scipy (0.19.0)
seaborn (0.7.1)
setuptools (27.2.0)
simplegeneric (0.8.1)
simplejson (3.10.0)
six (1.10.0)
snakeviz (0.4.1)
snowballstemmer (1.2.1)
Sphinx (1.5.6)
sphinx-rtd-theme (0.2.4)
SQLAlchemy (1.1.9)
terminado (0.6)
testpath (0.3)
tornado (4.5.1)
traitlets (4.3.2)
typing (3.6.1)
wcwidth (0.1.7)
wheel (0.29.0)
widgetsnbextension (2.0.0)
wrapt (1.10.10)

• pytest and operating system versions

OS : OSX El Captian 
pytest --version
This is pytest version 3.0.7, 
  pytest-instafail-0.3.0 at 
  pytest-rerunfailures-2.1.0 at 
  pytest-timeout-1.0.0 at

• Minimal example if possible

import pytest
import time


@pytest.mark.timeout(2)
@pytest.mark.flaky(3) # comment this line to see the test fail as expected
def test_new():
    time.sleep(10)
    assert True

Start by running the minimal example with pytest 3.0.7, pytest-rerunfailures 2.1.0, and pytest-timeout 1.0.0 to observe how the timeout behaves across retries. Trace the interaction between the two plugins and confirm that a test sleeping for 10 seconds with a 2-second timeout is interrupted even when marked for reruns.