tortoise / tortoise/tortoise-orm

Any request to DB results in error after running normally for some time

Open
#1,348 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.6k
Forks
516
Avg merge
2d 21h
Merged PRs (30d)
9

Description

Describe the bug
After application has been working for some time normally (about 30h in my case), all requests to database begun throwing AttributeError: 'NoneType' object has no attribute 'at_eof' with traceback:

  File "/usr/local/lib/python3.10/site-packages/tortoise/transactions.py", line 56, in wrapped
    async with in_transaction(connection_name):
               │              └ None
               └ <function in_transaction at 0x7facf703ab00>
  File "/usr/local/lib/python3.10/site-packages/tortoise/backends/base/client.py", line 270, in __aenter__
    self.connection._connection = await self.connection._parent._pool.acquire()
    │    │                              │    └ <member 'connection' of 'TransactionContextPooled' objects>
    │    │                              └ <tortoise.backends.base.client.TransactionContextPooled object at 0x7facf1fb8a00>
    │    └ <member 'connection' of 'TransactionContextPooled' objects>
    └ <tortoise.backends.base.client.TransactionContextPooled object at 0x7facf1fb8a00>
  File "/usr/local/lib/python3.10/site-packages/aiomysql/pool.py", line 142, in _acquire
    await self._fill_free_pool(True)
          │    └ <function Pool._fill_free_pool at 0x7facf3fa8ca0>
          └ <aiomysql.pool.Pool object at 0x7facf3e547c0>
  File "/usr/local/lib/python3.10/site-packages/aiomysql/pool.py", line 158, in _fill_free_pool
    if conn._reader.at_eof() or conn._reader.exception():
       │    │                   │    └ None
       │    │                   └ <aiomysql.connection.Connection object at 0x7facf32fe6e0>
       │    └ None
       └ <aiomysql.connection.Connection object at 0x7facf32fe6e0>

AttributeError: 'NoneType' object has no attribute 'at_eof'

To Reproduce
Write an application that sometimes handles requests (for example, Telegram bot) with Tortoise ORM and request handler decorated with @atomic and let it run for a while.

Expected behavior
Application keeps working correctly no matter how long ago it was launched.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the @atomic path in tortoise/transactions.py and the pooled connection handling shown in tortoise/backends/base/client.py, then compare it with aiomysql.pool.py in the traceback. Reproduce the long-running application scenario and verify that database requests still work after the idle period and connection failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, python
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.