database error with pytest-django and pytest-xdist 1.26.1

pytest with pytest-django is giving us errors tearing down databases since upgrading to pytest-xdist 1.26.1. This happens nearly 100% of the time, but very occasionally doesn't. I suspect that having a large number of tests exacerbates the problem.

We get an error like this (anonymised):

_________________________________________________________________________________________________________________________________________________________________ ERROR at teardown of TestClass.test_some_behaviour _________________________________________________________________________________________________________________________________________________________________
[gw12] linux2 -- Python 2.7.15 /home/user/virtualenv/venv/bin/python2

    def teardown_database():
        with django_db_blocker.unblock():
>           teardown_databases(db_cfg, verbosity=request.config.option.verbose)

../../virtualenv/venv/local/lib/python2.7/site-packages/pytest_django/fixtures.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../virtualenv/venv/local/lib/python2.7/site-packages/pytest_django/compat.py:14: in teardown_databases
    db_cfg
../../virtualenv/venv/local/lib/python2.7/site-packages/django/test/runner.py:509: in teardown_databases
    connection.creation.destroy_test_db(old_name, self.verbosity, self.keepdb)
../../virtualenv/venv/local/lib/python2.7/site-packages/django/db/backends/base/creation.py:264: in destroy_test_db
    self._destroy_test_db(test_database_name, verbosity)
../../virtualenv/venv/local/lib/python2.7/site-packages/django/db/backends/base/creation.py:283: in _destroy_test_db
    % self.connection.ops.quote_name(test_database_name))
../../virtualenv/venv/local/lib/python2.7/site-packages/django/db/backends/utils.py:64: in execute
    return self.cursor.execute(sql, params)
../../virtualenv/venv/local/lib/python2.7/site-packages/django/db/utils.py:95: in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <django.db.backends.utils.CursorWrapper object at 0x7f0c769703d0>, sql = 'DROP DATABASE "test_pytest_gw12"', params = None

    def execute(self, sql, params=None):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
>               return self.cursor.execute(sql)
E               OperationalError: database "test_pytest_gw12" is being accessed by other users
E               DETAIL:  There are 10 other sessions using the database.

../../virtualenv/venv/local/lib/python2.7/site-packages/django/db/backends/utils.py:62: OperationalError

I have been running with pytest-random and the --random argument to test this behaviour, and the name of the test that failed is different every time, and I don't think relevant to this behaviour; I don't think this is a problem with any of the tests. This problem occurs without the pytest-random plugin installed - I was just using that to convince myself that the problem wasn't related to a particular test suite.

Sometimes an additional failure with as IntegrityError will occur on the primary key of a record when inserting into the database, but I think this a symptom of the above error rather than a legitimate test error.

The commandline used is:

pytest --ds=test_settings_local --disable-warnings -n auto --random

This problem started occurring around the time pytest-xdist 1.26.1 was released, and downgrading to pytest-xdist 1.26.0 makes the problem (seem to) go away, which leads me to believe that the problem is with pytest-xdist 1.26.1

Start by running pytest --ds=test_settings_local --disable-warnings -n auto --random with pytest-xdist 1.26.1 and compare it with 1.26.0. Investigate the pytest-django database teardown path and the DROP DATABASE failure showing active sessions. Done means the distributed test run reliably tears down its databases without the reported OperationalError or secondary integrity failures.