pytest-dev / pytest-dev/pytest

Full diff thinks unittest.mock.ANY fails equality check in assertions

Open
#3,638 9 comments 15 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: help wanted type: enhancement type: feature-branch
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

When unittest.mock.ANY is used, and assertion fails, the pytest diff is confusing, and it mistakenly marks ANY being non-equal to the value on the other side. This is especially confusing with lists comparison. Simple example:

from unittest.mock import ANY
assert [{'id': 1, 'val': 5}] == [{'id': ANY, 'val': 4}]

The full diff produced with pytest -vv:

E       AssertionError: assert [{'id': 1, 'val': 5}] == [{'id': <ANY>, 'val': 4}]
E         At index 0 diff: {'id': 1, 'val': 5} != {'id': <ANY>, 'val': 4}
E         Full diff:
E         - [{'id': 1, 'val': 5}]
E         ?         ^         ^
E         + [{'id': <ANY>, 'val': 4}]
E         ?         ^^^^^         ^

In the full diff you can see that not only val, but also id is marked as being different, which is confusing, as it does not trigger the assertion. It would be a lot more clear if full diff only show the values actually comparing non-equal

pytest version: 3.6.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided unittest.mock.ANY reproducer and run it with pytest -vv to inspect the assertion output. Trace the full-diff generation from that failure and verify that equal values involving ANY are not marked as differences while the unequal val remains highlighted.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.