pytest-dev / pytest-dev/pytest
Assert error message is omitted with --tb=long if assertion has been made in method called by test method with.
Open
Nobody has claimed this yet.
topic: rewrite
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
- a detailed description of the bug or problem you are having
- output of
pip listfrom the virtual environment you are using - pytest and operating system versions
- minimal example if possible
pipi list:
anyio 3.6.2
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
arrow 1.2.3
asttokens 2.2.1
attrs 22.2.0
Babel 2.11.0
backcall 0.2.0
beautifulsoup4 4.11.2
bleach 6.0.0
brotlipy 0.7.0
certifi 2022.12.7
cffi 1.15.1
charset-normalizer 2.0.4
colorama 0.4.6
comm 0.1.2
contourpy 1.0.7
coverage 7.1.0
cryptography 38.0.4
cycler 0.11.0
debugpy 1.6.6
decorator 5.1.1
defusedxml 0.7.1
entrypoints 0.4
exceptiongroup 1.1.0
execnet 1.9.0
executing 1.2.0
fastjsonschema 2.16.2
flit_core 3.6.0
fonttools 4.38.0
fqdn 1.5.1
idna 3.4
importlib-metadata 6.0.0
iniconfig 2.0.0
ipykernel 6.21.1
ipython 8.9.0
ipython-genutils 0.2.0
ipywidgets 8.0.4
isoduration 20.11.0
jedi 0.18.2
Jinja2 3.1.2
json5 0.9.6
jsonpointer 2.3
jsonschema 4.17.3
jupyter 1.0.0
jupyter_client 8.0.2
jupyter-console 6.4.4
jupyter_core 5.2.0
jupyter-events 0.6.3
jupyter_server 2.2.1
jupyter_server_terminals 0.4.4
jupyterlab 3.5.3
jupyterlab-pygments 0.2.2
jupyterlab_server 2.16.5
jupyterlab-widgets 3.0.5
kiwisolver 1.4.4
lxml 4.9.1
MarkupSafe 2.1.2
matplotlib 3.6.3
matplotlib-inline 0.1.6
mistune 2.0.4
mkl-fft 1.3.1
mkl-random 1.2.2
mkl-service 2.4.0
nbclassic 0.5.1
nbclient 0.7.2
nbconvert 7.2.9
nbformat 5.7.3
nest-asyncio 1.5.6
notebook 6.5.2
notebook_shim 0.2.2
numpy 1.23.5
opencv-python 4.7.0.68
packaging 23.0
pandocfilters 1.5.0
parso 0.8.3
pep8 1.7.1
pickleshare 0.7.5
Pillow 9.4.0
pip 22.3.1
platformdirs 2.6.2
pluggy 1.0.0
ply 3.11
prometheus-client 0.16.0
prompt-toolkit 3.0.36
psutil 5.9.0
pure-eval 0.2.2
pycparser 2.21
Pygments 2.14.0
pyOpenSSL 22.0.0
pyparsing 3.0.9
PyQt5 5.15.7
PyQt5-sip 12.11.0
pyrsistent 0.19.3
PySocks 1.7.1
pytest 7.2.1
pytest-cache 1.0
pytest-cov 4.0.0
pytest-mock 3.10.0
pytest-pep8 1.0.6
python-dateutil 2.8.2
python-json-logger 2.0.4
pytz 2022.7
pywin32 305.1
pywinpty 2.0.10
PyYAML 6.0
pyzmq 25.0.0
qtconsole 5.4.0
QtPy 2.3.0
requests 2.28.1
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
Send2Trash 1.8.0
setuptools 65.6.3
sip 6.6.2
six 1.16.0
sniffio 1.3.0
soupsieve 2.3.2.post1
stack-data 0.6.2
terminado 0.17.1
tinycss2 1.2.1
toml 0.10.2
tomli 2.0.1
tornado 6.2
traitlets 5.9.0
typing_extensions 4.4.0
uri-template 1.2.0
urllib3 1.26.14
wcwidth 0.2.6
webcolors 1.12
webencodings 0.5.1
websocket-client 1.5.0
wheel 0.37.1
widgetsnbextension 4.0.5
win-inet-pton 1.1.0
wincertstore 0.2
zipp 3.12.0
Operating system version: Windows 11 Pro 22H2
PyTest Version 7.2.1
The assertion message is not printed in the output when an assertion with given message is made in a method that is not directly the test method.
Happens only with --tb=long
and of course with --tb=auto (default) on the first and last
The message is shown with --tb=short and even --tb=line
This obscures assert messages, which seams strange for the setting for longer, more verbose output.
Weirdly enough, if the called method excepts the assert and just raises a new assertion error with e.args the message IS shown. but also more unnecessary output.
I just notice in the summery the message appears again. Pycharm disables the summery per default, so i missed that.
Even still, it seams weird that the "long" setting is omitting the assert message in the trace.
Minimal example:
def assert_test_except():
try:
assert 2 == 4, "this is printed"
except AssertionError as e:
raise AssertionError(e.args)
def assert_test():
assert 2 == 4, "this is not printed"
def test_direct_assert():
assert 2 == 4, "this naturally is printed"
#TEST: message is printed, and addition exception is printed (as expected)
def test_assertion_messages_with_except():
assert_test_except()
#TEST: message in called method is NOT printed (only in summery)
def test_assertion_message_no_except():
assert_test()
Outputs:
C:\Users\marce\anaconda3\envs\monoRepo\python.exe C:/XXX/AppData/Local/JetBrains/Toolbox/apps/PyCharm-P/ch-0/223.8214.51/plugins/python/helpers/pycharm/_jb_pytest_runner.py --path C:\XXX\tests\test_pytest.py -- --jb-show-summary
Testing started at 12:24 ...
Launching pytest with arguments C:\Users\marce\Documents\monoRepo\processing_container\vectoriser\tests\test_pytest.py in C:\XXX\tests
============================= test session starts =============================
platform win32 -- Python 3.9.16, pytest-7.2.1, pluggy-1.0.0 -- C:\XXX\python.exe
cachedir: .pytest_cache
rootdir: C:\XXX\tests
plugins: anyio-3.5.0, cov-4.0.0, mock-3.10.0, pep8-1.0.6
collecting ... collected 3 items
test_pytest.py::test_direct_assert FAILED [ 33%]
test_pytest.py:11 (test_direct_assert)
2 != 4
Expected :4
Actual :2
<Click to see difference>
def test_direct_assert():
> assert 2 == 4, "this naturally is printed"
E AssertionError: this naturally is printed
E assert 2 == 4
test_pytest.py:13: AssertionError
test_pytest.py::test_assertion_messages_with_except FAILED [ 66%]
test_pytest.py:15 (test_assertion_messages_with_except)
2 != 4
Expected :4
Actual :2
<Click to see difference>
def assert_test_catch():
try:
> assert 2 == 4, "this is printed"
E AssertionError: this is printed
E assert 2 == 4
test_pytest.py:3: AssertionError
During handling of the above exception, another exception occurred:
def test_assertion_messages_with_except():
> assert_test_catch()
test_pytest.py:17:
test_pytest.py::test_assertion_message_no_except FAILED [100%]
test_pytest.py:19 (test_assertion_message_no_except)
2 != 4
Expected :4
Actual :2
<Click to see difference>
def test_assertion_message_no_except():
> assert_test()
test_pytest.py:21:
================================== FAILURES ===================================
_____________________________ test_direct_assert ______________________________
def test_direct_assert():
> assert 2 == 4, "this naturally is printed"
E AssertionError: this naturally is printed
E assert 2 == 4
test_pytest.py:13: AssertionError
_____________________ test_assertion_messages_with_except _____________________
def assert_test_catch():
try:
> assert 2 == 4, "this is printed"
E AssertionError: this is printed
E assert 2 == 4
test_pytest.py:3: AssertionError
During handling of the above exception, another exception occurred:
def test_assertion_messages_with_except():
> assert_test_catch()
test_pytest.py:17:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def assert_test_catch():
try:
assert 2 == 4, "this is printed"
except AssertionError as e:
> raise AssertionError(e.args)
E AssertionError: ('this is printed\nassert 2 == 4',)
test_pytest.py:5: AssertionError
______________________ test_assertion_message_no_except _______________________
def test_assertion_message_no_except():
> assert_test()
test_pytest.py:21:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def assert_test():
> assert 2 == 4, "this is not printed"
E AssertionError: this is not printed
E assert 2 == 4
test_pytest.py:9: AssertionError
=========================== short test summary info ===========================
FAILED test_pytest.py::test_direct_assert - AssertionError: this naturally is...
FAILED test_pytest.py::test_assertion_messages_with_except - AssertionError: ...
FAILED test_pytest.py::test_assertion_message_no_except - AssertionError: thi...
============================== 3 failed in 0.14s ==============================
Process finished with exit code 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the minimal reproducer in test_pytest.py and run it with --tb=long, comparing direct and indirect assertions. Trace the long traceback rendering for assert_test() and ensure the assertion message appears there as it does in the summary and shorter traceback modes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100