pytest-dev / pytest-dev/pytest
Full diff thinks unittest.mock.ANY fails equality check in assertions
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
When unittest.mock.ANY is used, and assertion fails, the pytest diff is confusing, and it mistakenly marks ANY being non-equal to the value on the other side. This is especially confusing with lists comparison. Simple example:
from unittest.mock import ANY
assert [{'id': 1, 'val': 5}] == [{'id': ANY, 'val': 4}]
The full diff produced with pytest -vv:
E AssertionError: assert [{'id': 1, 'val': 5}] == [{'id': <ANY>, 'val': 4}]
E At index 0 diff: {'id': 1, 'val': 5} != {'id': <ANY>, 'val': 4}
E Full diff:
E - [{'id': 1, 'val': 5}]
E ? ^ ^
E + [{'id': <ANY>, 'val': 4}]
E ? ^^^^^ ^
In the full diff you can see that not only val, but also id is marked as being different, which is confusing, as it does not trigger the assertion. It would be a lot more clear if full diff only show the values actually comparing non-equal
pytest version: 3.6.2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided unittest.mock.ANY reproducer and run it with pytest -vv to inspect the assertion output. Trace the full-diff generation from that failure and verify that equal values involving ANY are not marked as differences while the unequal val remains highlighted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100