pytest-dev / pytest-dev/pytest

empty_parameter_set_mark = xfail + `--runxfail` => crash

Open
#4,497 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: bug
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

This started as a "hmm... that's weird" and ended at a crash. go figure.

This is super low priority and more of a result of "pytest has sooo many options!"

Today I learned that this generates a single test which is always skipped

import pytest

@pytest.mark.parametrize(
    ('a', 'b'),
    # no cases defined yet
    (

    ),
)
def test(a, b):
    pass
$ pytest t2.py 
============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-4.0.1, py-1.7.0, pluggy-0.8.0
rootdir: /tmp/t, inifile:
collected 1 item                                                               

t2.py s                                                                  [100%]

========================== 1 skipped in 0.01 seconds ===========================

But with a certain set of arguments:

$ pytest --runxfail -o empty_parameter_set_mark=xfail t2.py
============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-4.0.1, py-1.7.0, pluggy-0.8.0
rootdir: /tmp/t, inifile:
collected 1 item                                                               

t2.py E                                                                  [100%]

==================================== ERRORS ====================================
________________________ ERROR at setup of test[a0-b0] _________________________

self = <CallInfo when='setup' exception: 'SubRequest' object has no attribute 'param'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x7fe789d24840>
when = 'setup', treat_keyboard_interrupt_as_exception = False

    def __init__(self, func, when, treat_keyboard_interrupt_as_exception=False):
        #: context of invocation: one of "setup", "call",
        #: "teardown", "memocollect"
        self.when = when
        self.start = time()
        try:
>           self.result = func()

venv/lib/python3.6/site-packages/_pytest/runner.py:211: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
venv/lib/python3.6/site-packages/_pytest/runner.py:193: in <lambda>
    lambda: ihook(item=item, **kwds),
venv/lib/python3.6/site-packages/pluggy/hooks.py:284: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
venv/lib/python3.6/site-packages/pluggy/manager.py:67: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
venv/lib/python3.6/site-packages/pluggy/manager.py:61: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
venv/lib/python3.6/site-packages/_pytest/runner.py:114: in pytest_runtest_setup
    item.session._setupstate.prepare(item)
venv/lib/python3.6/site-packages/_pytest/runner.py:381: in prepare
    col.setup()
venv/lib/python3.6/site-packages/_pytest/python.py:1442: in setup
    fixtures.fillfixtures(self)
venv/lib/python3.6/site-packages/_pytest/fixtures.py:297: in fillfixtures
    request._fillfixtures()
venv/lib/python3.6/site-packages/_pytest/fixtures.py:470: in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
venv/lib/python3.6/site-packages/_pytest/fixtures.py:517: in getfixturevalue
    return self._get_active_fixturedef(argname).cached_result[0]
venv/lib/python3.6/site-packages/_pytest/fixtures.py:540: in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
venv/lib/python3.6/site-packages/_pytest/fixtures.py:626: in _compute_fixture_value
    fixturedef.execute(request=subrequest)
venv/lib/python3.6/site-packages/_pytest/fixtures.py:927: in execute
    return hook.pytest_fixture_setup(fixturedef=self, request=request)
venv/lib/python3.6/site-packages/pluggy/hooks.py:284: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
venv/lib/python3.6/site-packages/pluggy/manager.py:67: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
venv/lib/python3.6/site-packages/pluggy/manager.py:61: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
venv/lib/python3.6/site-packages/_pytest/fixtures.py:969: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
venv/lib/python3.6/site-packages/_pytest/fixtures.py:828: in call_fixture_func
    res = fixturefunc(**kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

request = <SubRequest 'a' for <Function 'test[a0-b0]'>>

    def get_direct_param_fixture_func(request):
>       return request.param
E       AttributeError: 'SubRequest' object has no attribute 'param'

venv/lib/python3.6/site-packages/_pytest/fixtures.py:301: AttributeError
=========================== 1 error in 0.26 seconds ============================

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the parametrized test in t2.py with --runxfail and -o empty_parameter_set_mark=xfail. Inspect the fixture setup path around get_direct_param_fixture_func in _pytest/fixtures.py; done means the command no longer crashes and the empty parameter set retains the intended result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.