pytest-dev / pytest-dev/pytest

tox and pytest runs can behave differently when "-p something.conftest" is specified

Open
#3,326 19 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: config topic: fixtures type: bug
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Did a little bit of debugging and wanted to share my findings here:

I have an sdist muacrypt which contains a test_muacrypt package and a conftest file inside.
In another project i have a pytest.ini containing addopts = -p test_muacrypt.conftest. This makes all fixtures and command line options available in a normal pytest run. fine.

Problem: When i run tox the fixtures from that plugin are not loaded.
This is surprising from a pure user perspective (tox and pytest runs should not have such differences).

Turns out, the problematic code is in this FixtureManager method:

def pytest_plugin_registered(self, plugin):
    nodeid = None
    try:
        p = py.path.local(plugin.__file__)
    except AttributeError:
        pass
    else:
        # construct the base nodeid which is later used to check
        # what fixtures are visible for particular tests (as denoted
        # by their test id)
        if p.basename.startswith("conftest.py"):
            nodeid = p.dirpath().relto(self.config.rootdir)
            if p.sep != nodes.SEP:
                nodeid = nodeid.replace(p.sep, nodes.SEP)
    self.parsefactories(plugin, nodeid)

in my case, test_muacrypt.conftest has a basename of conftest.py and it's path (inside .tox) is relative to the project root. So the plugin gets a "nodeid" and lookups for fixtures from that plugin fail, because the requesting side will not be a child id of that '.tox/py27/.../conftest.py`. Without tox, Plugins which reside outside a project rootdir will get an empty nodeid and are thus found properly.

Not sure how to fix this in pytest properly yet.

A work aground for me is to put the plugin into test_muacrypt.testing_plugin or so and import everything from the conftest file there.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining FixtureManager.pytest_plugin_registered and parsefactories, especially how conftest.py paths under tox are converted into nodeids. Reproduce the difference between a normal pytest run and tox with -p test_muacrypt.conftest, then verify that fixtures and command-line options load consistently in both environments.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.