pytest-dev / pytest-dev/pytest-django
Prioritizing pytests --first flags over Django's "usual" test order breaks mix of TestCase und TransactionTestCase with data migrations
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 367
- PR merge metrics
- No merged PRs in 30d
Description
This was introduced in #819. If I have data migrations in my project and two test cases, one TestCase and one TransactionTestCase, prioritizing the first flags over the order introduced in #214 can break the TestCase.
Running tests with --new-first and modifying the TransactionTestCase after the TestCase results in an empty database for the latter. If the project uses data migrations to create default data in the database this breaks the TestCase relying on that data. Running tests with --failed-first does the same if the TransactionTestCase is the last to have failed.
This means for a project using data migrations in this way and having both TestCases and TransactionTestCases the first flags cannot be used reliably currently.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the mixed TestCase and TransactionTestCase scenario with data migrations, using --new-first and --failed-first. Review the behavior introduced in #819 and the test-ordering change from #214; done means both flags preserve the data needed by the TestCase regardless of which case runs first or last.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- databases, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100