pytest-dev / pytest-dev/pytest

text diffs on huge files are slow

Open
#12,406 2 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: rewrite type: performance
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Ok, of course they are... But usually* when a dev commits the test, the test is passing, so the dev may not notice how preposterous a diff they are inadvertently asking for.

In my case, a 5 year old test happened to be comparing text files about 2million lines long as strings, functionally:

    assert fh1.read() == fh2.read()

This was fine, until the order of some fields changed and the test started hanging in CI for hours. The right thing to do is to fix this annoying test, but I thought that it might also make sense to push a fix up to pytest.

tl;dr - _diff_text() already knows the verbosity level - would it make sense to truncate the length of the diff calculated in "non verbose" mode? By default, the diff is truncated to the first 10 lines, so _diff_text() is doing extra computation that the caller will never see or use.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating pytest's _diff_text() implementation and tracing how non-verbose output is produced. Reproduce the issue with a comparison of very large text strings, then verify that non-verbose mode avoids unnecessary diff computation while still producing the expected first 10 lines of output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.