pytest-dev / pytest-dev/pytest-django

Circular dependency in TEST[DEPENDENCIES] arises after upgrade from 4.10.0 => 4.11.0

Open
#1,221 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.5k
Forks
367
PR merge metrics
No merged PRs in 30d

Description

When I upgrade from 4.10.0 => 4.11.0, the following error starts occurring

_______________________ ERROR at setup of test_foo ________________________

request = <SubRequest '_django_db_marker' for <Function test_foo>>

    @pytest.fixture(autouse=True)
    def _django_db_marker(request: pytest.FixtureRequest) -> None:
        """Implement the django_db marker, internal to pytest-django."""
        marker = request.node.get_closest_marker("django_db")
        if marker:
>           request.getfixturevalue("_django_db_helper")

.venv/lib/python3.11/site-packages/pytest_django/plugin.py:552: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.11/site-packages/pytest_django/fixtures.py:198: in django_db_setup
    db_cfg = setup_databases(
.venv/lib/python3.11/site-packages/django/test/utils.py:187: in setup_databases
    test_databases, mirrored_aliases = get_unique_databases_and_mirrors(aliases)
.venv/lib/python3.11/site-packages/django/test/utils.py:369: in get_unique_databases_and_mirrors
    test_databases = dict(dependency_ordered(test_databases.items(), dependencies))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

test_databases = dict_items([(('localhost', 5433, 'django.db.backends.postgresql', 'test_postgres'), ('postgres', ['myseconddb']))])
dependencies = {'myseconddb': ['default']}

    def dependency_ordered(test_databases, dependencies):
        """
        Reorder test_databases into an order that honors the dependencies
        described in TEST[DEPENDENCIES].
        """
        ordered_test_databases = []
        resolved_databases = set()
    
        # Maps db signature to dependencies of all its aliases
        dependencies_map = {}
    
        # Check that no database depends on its own alias
        for sig, (_, aliases) in test_databases:
            all_deps = set()
            for alias in aliases:
                all_deps.update(dependencies.get(alias, []))
            if not all_deps.isdisjoint(aliases):
                raise ImproperlyConfigured(
                    "Circular dependency: databases %r depend on each other, "
                    "but are aliases." % aliases
                )
            dependencies_map[sig] = all_deps
    
        while test_databases:
            changed = False
            deferred = []
    
            # Try to find a DB that has all its dependencies met
            for signature, (db_name, aliases) in test_databases:
                if dependencies_map[signature].issubset(resolved_databases):
                    resolved_databases.update(aliases)
                    ordered_test_databases.append((signature, (db_name, aliases)))
                    changed = True
                else:
                    deferred.append((signature, (db_name, aliases)))
    
            if not changed:
>               raise ImproperlyConfigured("Circular dependency in TEST[DEPENDENCIES]")
E               django.core.exceptions.ImproperlyConfigured: Circular dependency in TEST[DEPENDENCIES]

.venv/lib/python3.11/site-packages/django/test/utils.py:312: ImproperlyConfigured

My test case looks like this

@pytest.mark.django_db(databases=["myseconddb"])
def test_foo():
    assert True

My databases look like this

DATABASES = {
    "default": {
        "ENGINE": "django.contrib.gis.db.backends.postgis",
        "NAME": "postgres",
        "USER": "postgres",
        "PASSWORD": "postgres",
        "HOST": "localhost",
        "PORT": 5432,
    },
    "myseconddb": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "postgres",
        "USER": "postgres",
        "PASSWORD": "postgres",
        "HOST": "localhost",
        "PORT": 5433,
    },
    "myseconddb-replica": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "postgres",
        "USER": "postgres",
        "PASSWORD": "postgres",
        "HOST": "localhost",
        "PORT": 5433,
        "TEST": {
            "MIRROR": "myseconddb",
        },
    },
}

My test session

python -m pytest --durations=5  --ds myproject.settings_test -vvx -k test_foo path/to/foo/test_foo.py
============================= test session starts ==============================
platform darwin -- Python 3.11.11, pytest-7.4.4, pluggy-1.5.0 -- /Users/me/workspace/myproject/.venv/bin/python
cachedir: .pytest_cache
django: version: 4.2.20, settings: myproject.settings_test (from option)
rootdir: /Users/micahsmith/workspace/myproject
configfile: pyproject.toml
plugins: allure-pytest-2.14.2, asyncio-0.23.8, freezegun-0.4.2, anyio-4.9.0, socket-0.7.0, Faker-37.1.0, langsmith-0.3.31, myproject-pytest-plugin-0.1, django-4.11.0, mock-3.14.0, xdist-3.6.1
asyncio: mode=Mode.STRICT

I suspect this is related to https://github.com/pytest-dev/pytest-django/commit/8000db04f07822861331d0df8ef52f9c67eafc00

If I tweak my test case

@pytest.mark.django_db(databases=["default", "myseconddb"])
def test_foo():
    assert True

Then it passes

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure through the pytest-django django_db marker and inspect Django's django.test.utils.dependency_ordered(), especially the database aliases, mirrors, and TEST[DEPENDENCIES] shown in the traceback. Add a regression case for databases=["myseconddb"] while preserving the existing passing case that includes "default", then verify pytest completes without the circular-dependency error.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, postgresql, python
Domain
databases, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.