pytest-dev / pytest-dev/pytest-bdd

Excessive tracebacks

Open
#297 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.5k
Forks
251
Avg merge
43m
Merged PRs (30d)
2

Description

I'm struggling to get tracebacks in pytest-bdd tests down to a manageable size. --tb=short doesn't do too much. It seems to be worst on given steps.

Even on a simple failure within my given step, there are 12 lines of traceback, 3 of pytest-bdd internals and 9 of pytest internals. Is there something pytest-bdd can do to suppress these from the output?

example.feature

Scenario: Checking tracebacks
    Given a scenario
    When it fails
    Then there should be a readable traceback

test_example.py

from pytest_bdd import scenario, given, when, then, parsers
@scenario('example.feature', 'Checking tracebacks')
def test_traceback():
    pass
@given(parsers.parse('a {noun:w}'))
def given(noun):
    assert noun == 'banana'

@when('it fails')
def when():
    pass

@then(parsers.parse('there should be a {adjective:w} traceback'))
def then(adjective):
    pass

Output

(bdd-examples-project) bjl@BJL-LAP:/mnt/c/Users/bjl/wsl/bdd-examples-project/tests/features$ pytest -k 'test_traceback -traceback2' --tb=short

============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.6.7, pytest-4.4.1, py-1.8.0, pluggy-0.9.0
rootdir: /mnt/c/Users/bjl/wsl/bdd-examples-project/tests, inifile: pytest.ini
plugins: bdd-3.1.0
collected 26 items / 25 deselected / 1 selected

steps/test_example.py F                                                                                                                                                                                                               [100%]

================================================================================================================= FAILURES ==================================================================================================================
______________________________________________________________________________________________________________ test_traceback _______________________________________________________________________________________________________________
steps/test_example.py:32: in test_traceback
    def test_traceback():
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/pytest_bdd/scenario.py:195: in _execute_scenario
    _execute_step_function(request, scenario, step, step_func)
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/pytest_bdd/scenario.py:136: in _execute_step_function
    step_func(**kwargs)
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/pytest_bdd/steps.py:164: in step_func
    result = request.getfixturevalue(func.__name__)
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/_pytest/fixtures.py:478: in getfixturevalue
    return self._get_active_fixturedef(argname).cached_result[0]
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/_pytest/fixtures.py:501: in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/_pytest/fixtures.py:586: in _compute_fixture_value
    fixturedef.execute(request=subrequest)
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/_pytest/fixtures.py:895: in execute
    return hook.pytest_fixture_setup(fixturedef=self, request=request)
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/pluggy/hooks.py:289: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/pluggy/manager.py:68: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/pluggy/manager.py:62: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/_pytest/fixtures.py:937: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
/home/bjl/.local/share/virtualenvs/bdd-examples-project-dSboamh_/lib/python3.6/site-packages/_pytest/fixtures.py:794: in call_fixture_func
    res = fixturefunc(**kwargs)
steps/test_example.py:38: in given
    assert noun == 'banana'
E   AssertionError: assert 'scenario' == 'banana'
E     - scenario
E     + banana

(The long paths to pipenv's virtualenv are also certainly not helping...)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure using example.feature and test_example.py, then inspect the pytest-bdd frames shown from scenario.py and steps.py. Compare the traceback with pytest's --tb=short behavior; done means user step failures retain the useful application frame while suppressing unnecessary framework frames.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.