pytest-dev / pytest-dev/pytest-django

django.db.utils.InterfaceError: connection already closed when running pytest

Open
#1,099 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.5k
Forks
367
PR merge metrics
No merged PRs in 30d

Description

environment:

Postgresql 14.7.0
Python 3.10
Django==4.2.8
djangorestframework==3.14.0
pytest-django==4.7.0
pytest==7.4.3
pytest-celery==0.0.0
pytest-django==4.7.0
pytest-elasticsearch==4.0.2
pytest-factoryboy==2.6.0
pytest-mock==3.12.0
pytest-xdist==3.5.0

the error log

self = <django.contrib.sessions.backends.db.SessionStore object at 0x7f28520365f0>
no_load = False

    def _get_session(self, no_load=False):
        """
        Lazily load session from storage (unless "no_load" is True, when only
        an empty dict is stored) and store it in the current instance.
        """
        self.accessed = True
        try:
>           return self._session_cache
E           AttributeError: 'SessionStore' object has no attribute '_session_cache'

/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/base.py:187: AttributeError

During handling of the above exception, another exception occurred:

self = <DatabaseWrapper vendor='postgresql' alias='default'>, name = None

    def _cursor(self, name=None):
        self.close_if_health_check_failed()
        self.ensure_connection()
        with self.wrap_database_errors:
>           return self._prepare_cursor(self.create_cursor(name))

/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py:308: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py:26: in inner
    return func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DatabaseWrapper vendor='postgresql' alias='default'>, name = None

    @async_unsafe
    def create_cursor(self, name=None):
        if name:
            # In autocommit mode, the cursor will be used outside of a
            # transaction, hence use a holdable cursor.
            cursor = self.connection.cursor(
                name, scrollable=False, withhold=self.connection.autocommit
            )
        else:
>           cursor = self.connection.cursor()
E           psycopg2.InterfaceError: connection already closed

/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/base.py:330: InterfaceError

The above exception was the direct cause of the following exception:

client = <django.test.client.Client object at 0x7f28b0a61c30>
django_user_model = <class 'django.contrib.auth.models.User'>

    @pytest.mark.django_db
    def test_auth_view(client, django_user_model):
        user = django_user_model.objects.create_user("test", "test1234", is_staff=True, is_superuser=True)
        client.force_login(user)
        url = reverse('admin:auth_user_add')
>       resp = client.post(url, data={"username": "test2", "password1": "1qaz@WSX3edc", "password2": "1qaz@WSX3edc"}, follow=True)

notification/tests/test_admin.py:63: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.10/site-packages/django/test/client.py:957: in post
    response = self._handle_redirects(
/usr/local/lib/python3.10/site-packages/django/test/client.py:1173: in _handle_redirects
    response = request_method(
/usr/local/lib/python3.10/site-packages/django/test/client.py:927: in get
    response = super().get(path, data=data, secure=secure, headers=headers, **extra)
/usr/local/lib/python3.10/site-packages/django/test/client.py:457: in get
    return self.generic(
/usr/local/lib/python3.10/site-packages/django/test/client.py:609: in generic
    return self.request(**r)
/usr/local/lib/python3.10/site-packages/django/test/client.py:891: in request
    self.check_exception(response)
/usr/local/lib/python3.10/site-packages/django/test/client.py:738: in check_exception
    raise exc_value
/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py:55: in inner
    response = get_response(request)
/usr/local/lib/python3.10/site-packages/django/utils/deprecation.py:136: in __call__
    response = self.process_response(request, response)
activity/middleware.py:44: in process_response
    user_id = request.user.id
/usr/local/lib/python3.10/site-packages/django/utils/functional.py:266: in inner
    self._setup()
/usr/local/lib/python3.10/site-packages/django/utils/functional.py:419: in _setup
    self._wrapped = self._setupfunc()
/usr/local/lib/python3.10/site-packages/django/contrib/auth/middleware.py:25: in <lambda>
    request.user = SimpleLazyObject(lambda: get_user(request))
/usr/local/lib/python3.10/site-packages/django/contrib/auth/middleware.py:11: in get_user
    request._cached_user = auth.get_user(request)
/usr/local/lib/python3.10/site-packages/django/contrib/auth/__init__.py:191: in get_user
    user_id = _get_user_session_key(request)
/usr/local/lib/python3.10/site-packages/django/contrib/auth/__init__.py:60: in _get_user_session_key
    return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/base.py:53: in __getitem__
    return self._session[key]
/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/base.py:192: in _get_session
    self._session_cache = self.load()
/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/db.py:42: in load
    s = self._get_session_from_db()
/usr/local/lib/python3.10/site-packages/django/contrib/sessions/backends/db.py:32: in _get_session_from_db
    return self.model.objects.get(
/usr/local/lib/python3.10/site-packages/django/db/models/manager.py:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:633: in get
    num = len(clone)
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:380: in __len__
    self._fetch_all()
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:1881: in _fetch_all
    self._result_cache = list(self._iterable_class(self))
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:91: in __iter__
    results = compiler.execute_sql(
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:1560: in execute_sql
    cursor = self.connection.cursor()
/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py:26: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py:330: in cursor
    return self._cursor()
/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py:307: in _cursor
    with self.wrap_database_errors:
/usr/local/lib/python3.10/site-packages/django/db/utils.py:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py:308: in _cursor
    return self._prepare_cursor(self.create_cursor(name))
/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py:26: in inner
    return func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DatabaseWrapper vendor='postgresql' alias='default'>, name = None

    @async_unsafe
    def create_cursor(self, name=None):
        if name:
            # In autocommit mode, the cursor will be used outside of a
            # transaction, hence use a holdable cursor.
            cursor = self.connection.cursor(
                name, scrollable=False, withhold=self.connection.autocommit
            )
        else:
>           cursor = self.connection.cursor()
E           django.db.utils.InterfaceError: connection already closed

/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/base.py:330: InterfaceError

the test case

@pytest.mark.django_db
def test_auth_view(client, django_user_model):
    user = django_user_model.objects.create_user(
        "test", "test1234", is_staff=True, is_superuser=True
    )
    client.force_login(user)
    url = reverse("admin:auth_user_add")
    resp = client.post(
        url,
        data={"username": "test2", "password1": "1qaz@WSX3edc", "password2": "1qaz@WSX3edc"},
        follow=True,
    )
    assert resp.status_code == 200
    assert User.objects.filter(username="test2").exists()

It seems like the connection is close by some one, probably client, before the function is over.
the test case can pass if I use the @pytest.mark.django_db(transaction=True) marker. But I don't know the reason.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with notification/tests/test_admin.py and compare the test_auth_view behavior with and without the transaction=True marker around client.post. Inspect pytest-django transaction handling and the Django test client/session path shown in the traceback. Done means reproducing the connection-closure cause and confirming a fix or documented explanation.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, postgresql, python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.