tortoise / tortoise/tortoise-orm

Support pytest verbs test-system (was: Unable to rollback transactions using pytest and in_transaction context manager)

Open
#286 8 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
5.6k
Forks
516
Avg merge
2d 21h
Merged PRs (30d)
9

Description

Describe the bug
I wrote a pytest fixture that should be automatically run for each test. This fixture starts a new transaction using the in_transaction context manager, yields, and then tries to roll back. But the fixture breaks when it tries to roll back with an error that the token was created in a different context like this:

ValueError: <Token var=<ContextVar name='default' default=<tortoise.backends.asyncpg.client.AsyncpgDBClient object at 0x7f8c294a6670> at 0x7f8c294b2810> at 0x7f8c285a7140> was created in a different Context

To Reproduce

  1. Install pytest, pytest-asyncio, tortoise-orm and asyncpg.
  2. Start the PostgreSQL server and create a new table with the contents:
CREATE TABLE mymodelfortest (
	id VARCHAR(255) PRIMARY KEY,
	name TEXT DEFAULT 'text'
);
  1. Create a new test_tortoise.py test file:
import contextlib

import pytest
from tortoise import Tortoise, fields, models
from tortoise.transactions import in_transaction


class MyModelForTest(models.Model):
    id = fields.CharField(pk=True, max_length=255)
    name = fields.TextField(default="text")


@pytest.fixture(autouse=True)
async def init_db():
    await Tortoise.init(
        db_url="postgres://postgres:postgres@localhost:5432/postgres",
        modules={"models": ["test_tortoise"]},
    )
    yield
    await Tortoise.close_connections()


@pytest.fixture(autouse=True)
async def db_transaction(init_db):
    with contextlib.suppress(RuntimeError):
        async with in_transaction():
            yield
            raise RuntimeError


@pytest.mark.asyncio
async def test_a():
    await MyModelForTest.create(id="test")
    assert (await MyModelForTest.all()) == [MyModelForTest(id="test")]


@pytest.mark.asyncio
async def test_b():
    assert (await MyModelForTest.all()) == []

  1. Run pytest and get a message about tests teardown errors and some failed tests:
$ pytest test_tortoise.py
====================================================================================== test session starts ======================================================================================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /home/nik/CodeSources/tortoise-test
plugins: asyncio-0.10.0
collected 2 items                                                                                                                                                                               

test_tortoise.py .EFE                                                                                                                                                                     [100%]

============================================================================================ ERRORS =============================================================================================
__________________________________________________________________________________ ERROR at teardown of test_a __________________________________________________________________________________

init_db = None

    @pytest.fixture(autouse=True)
    async def db_transaction(init_db):
        with contextlib.suppress(RuntimeError):
            async with in_transaction():
                yield
>               raise RuntimeError
E               RuntimeError

test_tortoise.py:28: RuntimeError

During handling of the above exception, another exception occurred:

init_db = None

    @pytest.fixture(autouse=True)
    async def db_transaction(init_db):
        with contextlib.suppress(RuntimeError):
            async with in_transaction():
                yield
>               raise RuntimeError

test_tortoise.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <tortoise.backends.base.client.TransactionContextPooled object at 0x7f8c28e240a0>, exc_type = <class 'RuntimeError'>, exc_val = RuntimeError()
exc_tb = <traceback object at 0x7f8c285bc1c0>

    async def __aexit__(self, exc_type, exc_val, exc_tb) -> None:
        if not self.connection._finalized:
            if exc_type:
                # Can't rollback a transaction that already failed.
                if exc_type is not TransactionManagementError:
                    await self.connection.rollback()
            else:
                await self.connection.commit()
>       current_transaction_map[self.connection_name].reset(self.token)
E       ValueError: <Token var=<ContextVar name='default' default=<tortoise.backends.asyncpg.client.AsyncpgDBClient object at 0x7f8c294a6670> at 0x7f8c294b2810> at 0x7f8c285a7140> was created in a different Context

../../.virtualenvs/tortoise-test-aUW8XuHC-py3.8/lib/python3.8/site-packages/tortoise/backends/base/client.py:173: ValueError
__________________________________________________________________________________ ERROR at teardown of test_b __________________________________________________________________________________

init_db = None

    @pytest.fixture(autouse=True)
    async def db_transaction(init_db):
        with contextlib.suppress(RuntimeError):
            async with in_transaction():
                yield
>               raise RuntimeError
E               RuntimeError

test_tortoise.py:28: RuntimeError

During handling of the above exception, another exception occurred:

init_db = None

    @pytest.fixture(autouse=True)
    async def db_transaction(init_db):
        with contextlib.suppress(RuntimeError):
            async with in_transaction():
                yield
>               raise RuntimeError

test_tortoise.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <tortoise.backends.base.client.TransactionContextPooled object at 0x7f8c284e7100>, exc_type = <class 'RuntimeError'>, exc_val = RuntimeError()
exc_tb = <traceback object at 0x7f8c29516c00>

    async def __aexit__(self, exc_type, exc_val, exc_tb) -> None:
        if not self.connection._finalized:
            if exc_type:
                # Can't rollback a transaction that already failed.
                if exc_type is not TransactionManagementError:
                    await self.connection.rollback()
            else:
                await self.connection.commit()
>       current_transaction_map[self.connection_name].reset(self.token)
E       ValueError: <Token var=<ContextVar name='default' default=<tortoise.backends.asyncpg.client.AsyncpgDBClient object at 0x7f8c28e155b0> at 0x7f8c285d8e00> at 0x7f8c284db680> was created in a different Context

../../.virtualenvs/tortoise-test-aUW8XuHC-py3.8/lib/python3.8/site-packages/tortoise/backends/base/client.py:173: ValueError
=========================================================================================== FAILURES ============================================================================================
____________________________________________________________________________________________ test_b _____________________________________________________________________________________________

    @pytest.mark.asyncio
    async def test_b():
>       assert (await MyModelForTest.all()) == []
E       assert [<MyModelForTest: test>] == []
E         Left contains one more item: <MyModelForTest: test>
E         Use -v to get the full diff

test_tortoise.py:39: AssertionError
============================================================================ 1 failed, 1 passed, 2 errors in 20.18s =============================================================================

Expected behavior
Transactions should be able to roll back after the test is completed and the fixture tries to teardown, and all tests must pass.

Additional context
OS: Linux 5.5.2-arch1-1
PostgreSQL: 12.1-alpine for docker
Docker version: 19.03.5-ce
Python: 3.8.1
Dependencies:

aiosqlite==0.11.0
asyncpg==0.20.1
attrs==19.3.0
ciso8601==2.1.3
more-itertools==8.2.0
packaging==20.1
pluggy==0.13.1
py==1.8.1
pyparsing==2.4.6
PyPika==0.35.21
pytest==5.3.5
pytest-asyncio==0.10.0
six==1.14.0
tortoise-orm==0.15.9
typing-extensions==3.7.4.1
wcwidth==0.1.8

I previously used tortoise-orm 0.14.1 and solved this issue using this fixture:

@pytest.fixture(autouse=True)
async def db_transaction():
    transaction = await start_transaction()
    yield
    await transaction.rollback()

But since start_transaction is deprecated, so I tried changing it for the in_transaction context manager and ran into this problem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the fixture in test_tortoise.py with pytest, then inspect TransactionContextPooled.aexit in tortoise/backends/base/client.py around line 173. Done means the transaction rolls back during fixture teardown, test_b sees no rows, and pytest test_tortoise.py completes without teardown errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
database, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.