pytest-dev / pytest-dev/pytest

Wrong session scoped fixtures parametrization

Open
#2,844 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

topic: fixtures type: bug
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Consider following testing code:

import pytest


@pytest.fixture(scope="session")
def session_wrapper(param1, param2):
    return param1, param2


class TestClass:
    @pytest.fixture(scope="session",
                    params=["p1 inside A", "p1 inside B"])
    def param1(self, request):
        return request.param

    @pytest.fixture(scope="session")
    def param2(self):
        return "p2 inside"

    def test_inside(self, session_wrapper, param1, param2):
        assert session_wrapper == (param1, param2)


@pytest.fixture(scope="session",
                params=["p1 outside A", "p1 outside B"])
def param1(request):
    return request.param


@pytest.fixture(scope="session")
def param2():
    return "p2 outside"


def test_outside(session_wrapper, param1, param2):
    assert session_wrapper == (param1, param2)

Session-scoped fixture session_wrapper is parametrized by param1 and param2 fixtures, which the fixture returns as tuple. Tests check if session_wrapper really returns current param1 and param2 instances. Which is not the case - first session_wrapper instance in new context returns wrong params (from previous context):

$ py.test -vv test_f.py
============================= test session starts =============================
platform win32 -- Python 3.4.3, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 -- c:\program files (x86)\python34\python.exe
cachedir: .cache
rootdir: D:\workspace\businessFactory\calistos, inifile: setup.cfg
plugins: cov-2.5.1
collecting ... collected 4 items

test_f.py::TestClass::test_inside[p1 inside A] PASSED
test_f.py::test_outside[p1 outside A] FAILED
test_f.py::TestClass::test_inside[p1 inside B] PASSED
test_f.py::test_outside[p1 outside B] PASSED

================================== FAILURES ===================================
_________________________ test_outside[p1 outside A] __________________________
test_f.py:37: in test_outside
    assert session_wrapper == (param1, param2)
E   AssertionError: assert ('p1 inside', 'p2 inside') == ('p1 outside A', 'p2 outside')
E     At index 0 diff: 'p1 inside' != 'p1 outside A'
E     Full diff:
E     - ('p1 inside', 'p2 inside')
E     ?      ^^           ^^
E     + ('p1 outside A', 'p2 outside')
E     ?      ^^^    ++       ^^^
===================== 1 failed, 3 passed in 0.13 seconds ======================
Coverage.py warning: No data was collected. (no-data-collected)

If TestClass is at the end of the script (first outside test and context, then inside), equivalent result is achieved:

test_f.py::test_outside[p1 outside A] PASSED
test_f.py::TestClass::test_inside[p1 inside A] FAILED
test_f.py::test_outside[p1 outside B] PASSED
test_f.py::TestClass::test_inside[p1 inside B] PASSED

When session_wrapper has class or function scope, everything works as expected.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with the provided test_f.py example and compare the two test-ordering cases. Then trace pytest's session-scoped fixture parametrization and context handling; done means both inside and outside tests receive their current param1 and param2 values regardless of collection order.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.