pytest-dev / pytest-dev/pytest-html
Pytest always outputs traceback for XFAIL [NOTRUN] test
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 779
- Forks
- 260
- PR merge metrics
- No merged PRs in 30d
Description
Hi!
Is there anybody who can elaborate what is going wrong with pytest-html, which generates results for XFailed tests on 'setup' without running the actual test? It is always printing this traceback in details:
self = <CallInfo when='setup' exception: [NOTRUN] Failed due to changes in functionality>, func = <function call_runtest_hook.<locals>.<lambda> at 0x7fc6c50d9f28>, when = 'setup'
def __init__(self, func, when):
#: context of invocation: one of "setup", "call",
#: "teardown", "memocollect"
self.when = when
self.start = time()
try:
> self.result = func()
.venv/lib/python3.5/site-packages/_pytest/runner.py:163:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv/lib/python3.5/site-packages/_pytest/runner.py:151: in <lambda>
return CallInfo(lambda: ihook(item=item, **kwds), when=when)
.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:745: in __call__
return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:339: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:334: in <lambda>
_MultiCall(methods, kwargs, hook.spec_opts).execute()
.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:613: in execute
return _wrapped_call(hook_impl.function(*args), self.execute)
.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:254: in _wrapped_call
return call_outcome.get_result()
.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:279: in get_result
raise ex[1].with_traceback(ex[2])
.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:265: in __init__
self.result = func()
.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py:614: in execute
res = hook_impl.function(*args)
.venv/lib/python3.5/site-packages/_pytest/skipping.py:190: in pytest_runtest_setup
check_xfail_no_run(item)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
item = <Function 'test_send_sync_with_distinct_sessions'>
def check_xfail_no_run(item):
"""check xfail(run=False)"""
if not item.config.option.runxfail:
evalxfail = item._evalxfail
if evalxfail.istrue():
if not evalxfail.get('run', True):
> pytest.xfail("[NOTRUN] " + evalxfail.getexplanation())
E _pytest.skipping.XFailed: [NOTRUN] Failed due to changes in functionality
.venv/lib/python3.5/site-packages/_pytest/skipping.py:208: XFailed
Mainly, I just wanna to see only the reason, which is XFailed: [NOTRUN] Failed due to changes in functionality, same as it works for SKIPPED tests
Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce an XFAIL(run=False) test in a pytest HTML report and compare its output with a skipped test. Start with the XFailed handling shown in _pytest/skipping.py and trace how the plugin renders setup outcomes. Done means the report shows only the XFailed reason instead of the full traceback.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- html, python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100