pytest-dev / pytest-dev/pytest-xdist

Fixture teardown doesnt always complete

Open
#537 8 comments 8 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

Hi,

I have a number of fixtures that sometimes can take a bit of time to finish both setting up and tearing down, I keep seeing inconsistencies in the teardown completing, i recreated the following simple test

if the sleep inside the teardown of the fixture is anything above 5 seconds, the file reflecting teardown-step2 is never created, Is this expected behavior? is there some sort of timeout on failure before the test execution is aborted completely in the worker after a failure?

The code

import pytest
import tempfile
import os
import logging
import time
logger = logging.getLogger(__file__)

@pytest.fixture(scope="class")
def cleandir():
    newpath = tempfile.mkdtemp()
    os.chdir(newpath)
    yield
    logger.warning("Doing Teardown")
    open("teardown-step1", "w")
    time.sleep(20)
    open("teardown-step2", "w")


@pytest.mark.usefixtures("cleandir")
class TestDirectoryInit:
    def test_cwd_starts_empty(self):
        assert os.listdir(os.getcwd()) == []
        with open("myfile", "w") as f:
            f.write("hello")

    def test_cwd_again_starts_empty(self):
        logger.info(os.getcwd())
        logger.info(os.listdir(os.getcwd()))
        assert os.listdir(os.getcwd()) == []

The output

$ pytest -s -v -x -n=5 --log-cli-level=INFO --dist=loadscope ./e2etest/tests/pydisttest1.py
============================= test session starts =============================
platform win32 -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1 -- c:\users\niroa\appdata\local\programs\python\python38\python.exe
cachedir: .pytest_cache
rootdir: E:\repos2\datareplication-service-nodms\datareplication-main-nodms
plugins: forked-1.1.3, logger-0.5.1, xdist-1.32.0
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I
[gw0] win32 Python 3.8.1 cwd: E:\repos2\xxx
[gw1] win32 Python 3.8.1 cwd: E:\repos2\xxx
[gw2] win32 Python 3.8.1 cwd: E:\repos2\xxx
[gw3] win32 Python 3.8.1 cwd: E:\repos2\xxx
[gw4] win32 Python 3.8.1 cwd: E:\repos2\xxx
[gw0] Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)]
[gw1] Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)]
[gw2] Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)]
[gw3] Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)]
[gw4] Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)]
gw0 [2] / gw1 [2] / gw2 [2] / gw3 [2] / gw4 [2]

scheduling tests via LoadScopeScheduling

e2etest/tests/pydisttest1.py::TestDirectoryInit::test_cwd_starts_empty
[gw0] PASSED e2etest/tests/pydisttest1.py::TestDirectoryInit::test_cwd_starts_empty
e2etest/tests/pydisttest1.py::TestDirectoryInit::test_cwd_again_starts_empty
[gw0] FAILED e2etest/tests/pydisttest1.py::TestDirectoryInit::test_cwd_again_starts_empty

================================== FAILURES ===================================
________________ TestDirectoryInit.test_cwd_again_starts_empty ________________
[gw0] win32 -- Python 3.8.1 c:\users\xxx\appdata\local\programs\python\python38\python.exe

self = <tests.pydisttest1.TestDirectoryInit object at 0x0000029D2BFFBA00>

    def test_cwd_again_starts_empty(self):
        logger.info(os.getcwd())
        logger.info(os.listdir(os.getcwd()))
>       assert os.listdir(os.getcwd()) == []
E       AssertionError: assert ['myfile'] == []
E         Left contains one more item: 'myfile'
E         Full diff:
E         - ['myfile']
E         + []

E:\repos2\xxx\e2etest\tests\pydisttest1.py:29: AssertionError
------------------------------ Captured log call ------------------------------
INFO     E:\repos2\xxx\e2etest\tests\pydisttest1.py:pydisttest1.py:27 C:\Users\xxx\AppData\Local\Temp\tmpu66u09mc
INFO     E:\repos2\xxx\e2etest\tests\pydisttest1.py:pydisttest1.py:28 ['myfile']
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!! xdist.dsession.Interrupted: stopping after 1 failures !!!!!!!!!!!!
======================== 1 failed, 1 passed in 11.12s =========================

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue from e2etest/tests/pydisttest1.py with pytest -s -v -x -n=5 and the shown fixture. Start by tracing the xdist worker interruption reported as xdist.dsession.Interrupted and how fixture teardown is handled after the first failure. Done means the expected behavior is established and covered for the teardown-step2 case, or the limitation is clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.