tortoise / tortoise/tortoise-orm
When using update_or_create in the presence of concurrent requests, it can lead to deadlocks.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 516
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 9
Description
Describe the bug
A clear and concise description of what the bug is.
Whilst employing tortoise-orm update_or_create within my FastAPI framework, I encountered a predicament wherein deadlocks arose during the execution of concurrent coroutines. I conducted several tests with tortoise-orm and encountered the subsequent error message:
Traceback (most recent call last):
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/models.py", line 1060, in get_or_create
await cls.select_for_update().filter(**kwargs).using_db(connection).get(),
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/queryset.py", line 1020, in _execute
raise DoesNotExist("Object does not exist")
tortoise.exceptions.DoesNotExist: Object does not exist
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/backends/mysql/client.py", line 44, in translate_exceptions_
return await func(self, *args)
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/backends/mysql/client.py", line 172, in execute_insert
await cursor.execute(query, values)
File "asyncmy/cursors.pyx", line 179, in execute
File "asyncmy/cursors.pyx", line 364, in _query
File "asyncmy/connection.pyx", line 450, in query
File "asyncmy/connection.pyx", line 638, in _read_query_result
File "asyncmy/connection.pyx", line 1025, in read
File "asyncmy/connection.pyx", line 600, in read_packet
File "asyncmy/protocol.pyx", line 190, in asyncmy.protocol.MysqlPacket.raise_for_error
File "asyncmy/protocol.pyx", line 194, in asyncmy.protocol.MysqlPacket.raise_for_error
File "asyncmy/errors.pyx", line 128, in asyncmy.errors.raise_mysql_exception
File "asyncmy/errors.pyx", line 137, in asyncmy.errors.raise_mysql_exception
asyncmy.errors.OperationalError: (1213, 'Deadlock found when trying to get lock; try restarting transaction')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/ddd/rr.py", line 42, in main
await asyncio.gather(*tasks)
File "/ddd/rr.py", line 23, in update_or_create_test
r = await Entry.update_or_create(param=param, defaults={"name": name})
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/models.py", line 1110, in update_or_create
return await cls.get_or_create(defaults, db, **kwargs)
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/models.py", line 1065, in get_or_create
return await cls.create(using_db=connection, **defaults, **kwargs), True
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/models.py", line 1136, in create
await instance.save(using_db=db, force_create=True)
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/models.py", line 944, in save
await executor.execute_insert(self)
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/backends/base/executor.py", line 227, in execute_insert
insert_result = await self.db.execute_insert(self.insert_query, values)
File "/Users/hubery/.pyenv/versions/ldcts/lib/python3.10/site-packages/tortoise/backends/mysql/client.py", line 52, in translate_exceptions_
raise OperationalError(exc)
tortoise.exceptions.OperationalError: (1213, 'Deadlock found when trying to get lock; try restarting transaction')
To Reproduce
Steps to reproduce the behavior, preferably a small code snippet.
To setup a quick mysql server I used docker:
docker run --rm -d --name some-mysql -e MYSQL_ROOT_PASSWORD=docker -e MYSQL_DATABASE=test -p 3307:3306 mysql:8.2.0
Then I installed tortoise-orm this way:
pip install tortoise-orm[asyncmy]==0.20.0
example code:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# @File : rr.py
# docker run --rm -d --name some-mysql -e MYSQL_ROOT_PASSWORD=docker -e MYSQL_DATABASE=test -p 3307:3306 mysql:8.2.0
import asyncio
import logging
from tortoise import Tortoise, fields, run_async
from tortoise.models import Model
import traceback
logging.basicConfig(level=logging.DEBUG)
class Entry(Model):
param = fields.TextField()
name = fields.TextField()
async def update_or_create_test(param, name):
r = await Entry.update_or_create(param=param, defaults={"name": name})
await asyncio.sleep(1)
async def main():
await Tortoise.init(
db_url="mysql://root:docker@127.0.0.1:3307/test",
modules={"models": ["__main__"]},
# _create_db=True # Use it only once at first run
)
await Tortoise.generate_schemas()
await Entry.all().delete()
tasks = []
for i in range(10):
param = f"param{i}"
name = f"mame{i}"
task = asyncio.ensure_future(update_or_create_test(param, name))
tasks.append(task)
try:
await asyncio.gather(*tasks)
except Exception:
print(traceback.format_exc())
finally:
await Tortoise.close_connections()
if __name__ == '__main__':
for i in range(10):
run_async(main())
Expected behavior
A clear and concise description of what you expected to happen.
Desirous of circumventing exceptions, or seeking guidance on the proper utilization of the create_or_update function, I humbly request your assistance.
Additional context
Add any other context about the problem here.
Python 3.10.6
tortoise-orm==0.20.0
asyncmy==0.2.8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The reported path enters tortoise/models.py at get_or_create and update_or_create, then the MySQL backend's client.py at execute_insert. Reproduce the concurrent calls in rr.py against the provided MySQL 8.2 Docker setup, then determine and document the supported behavior or required change for the reported deadlock.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, python
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100