pytest-dev / pytest-dev/pytest
Unequal location hints point at equal sections
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
I've encountered a situation where pytest incorrectly reports the character positions of mismatched characters in two strings being compared. Consider this test:
import textwrap
expected = textwrap.dedent("""
--w-r---wx 1 0 0 999999 2019-02-01 bar
drwxr-xr-x 2 0 0 64 2019-02-01 biz
-rw-r--r-- 1 0 0 100 2019-02-01 foo
""").lstrip().format(**locals())
actual = textwrap.dedent("""
--w-r---wx 1 0 0 999999 2019-02-01 bar
drwxrwxr-x 2 0 0 64 2019-02-01 biz
-rw-rw-r-- 1 0 0 100 2019-02-01 foo
""").lstrip().format(**locals())
def test_equal():
assert expected == actual
Running pytest on that file emits:
draft $ pip-run -q pytest -- -m pytest -vv test-cmp.py
================================================================================== test session starts ==================================================================================
platform darwin -- Python 3.7.2, pytest-4.2.0, py-1.7.0, pluggy-0.8.1 -- /Library/Frameworks/Python.framework/Versions/3.7/bin/python3
cachedir: .pytest_cache
rootdir: /Users/jaraco/draft, inifile:
plugins: xonsh-0.8.9
collected 1 item
test-cmp.py::test_equal FAILED [100%]
======================================================================================= FAILURES ========================================================================================
______________________________________________________________________________________ test_equal _______________________________________________________________________________________
def test_equal():
> assert expected == actual
E AssertionError: assert '--w-r---wx 1...9-02-01 foo\n' == '--w-r---wx 1 ...9-02-01 foo\n'
E --w-r---wx 1 0 0 999999 2019-02-01 bar
E - drwxr-xr-x 2 0 0 64 2019-02-01 biz
E ? ---
E + drwxrwxr-x 2 0 0 64 2019-02-01 biz
E ? +++
E - -rw-r--r-- 1 0 0 100 2019-02-01 foo
E ? ---
E + -rw-rw-r-- 1 0 0 100 2019-02-01 foo
E ? +++
test-cmp.py:17: AssertionError
=============================================================================== 1 failed in 0.06 seconds ================================================================================
The failure is expected, but the reporting that it emits is not. As you can see, the ^^^ hints are pointing to sections that are the same in the actual and expected results.
As I'd never encountered this before, I originally thought it was a unittest bug, but this example doesn't use unittest at all.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the reproducer in test-cmp.py with pytest and compare the displayed hints with the differing characters in the two multiline strings. Trace pytest's assertion-reporting path for string comparisons. Done means the location hints identify the mismatched sections rather than equal sections, with the reproducer still passing as an appropriate regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100