pytest-dev / pytest-dev/pytest-django

--junitxml report includes time spent on migrations for first test

Open
#1,012 0 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.5k
Forks
367
PR merge metrics
No merged PRs in 30d

Description

When using pytest's --junitxml flag to produce a test report, the timing information includes the migrations for the first test run which requires migrations. In our environment, we often randomize the order of test runs and split them based on past timing information. This doesn't work very well, because we can't measure the time taken to run the test when it includes the time spent on migrations – a time penalty we'll have to pay if any test uses the database fixture.

I'm not sure if there is an easy way to solve this -- but if there is, I'd happily take a stab at a PR.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing pytest-django's handling of the Django database fixture and migrations alongside pytest's --junitxml timing. Determine where migration time enters the first test's reported duration, then verify that the report measures the test without that setup penalty while preserving correct timing for later tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
databases, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.