pytest-dev / pytest-dev/pytest

Issues with nodeid consistency when pytest --pyargs is not run from the root directory

Open
#3,714 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: collection type: bug
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Consider the following (standard) test layout:

quux
└── pkg
    ├── __init__.py
    └── tests
        ├── conftest.py
        ├── __init__.py
        └── test_foo.py

where quux is in PYTHONPATH (e.g., add it directly to PYTHONPATH, or quux could actually be site-packages).

conftest.py is set to report the items nodeids:

def pytest_collection_modifyitems(session, config, items):
    for item in items:
        print(item.nodeid)

and test_foo contains a single test, test_bar

If one runs pytest --pyargs pkg, --pyargs pkg.tests, or pkg.tests.test_foo, from quux, then the nodeid is correctly reported as pkg/tests/test_foo.py::test_bar

If one runs the same commands from quux's parent, then the nodeids are reported as quux/tests/test_foo.py::test_bar -- i.e., relative to cwd rather than to where the tests have been found in the PYTHONPATH.

Finally, if one runs the same commands from a directory that is unrelated (not a parent) of quux, then the nodeids are reported as pkg/tests/test_foo.py::test_bar, tests/test_foo.py::test_bar, and test_foo.py::test_bar respectively -- i.e., relative to the parent of the last specified module.

I think the correct behavior is clearly to always report as relative to where the tests have been found in the PYTHONPATH, i.e. pkg/tests/test_foo.py::test_bar.

Tested from a fresh Py3.6 (Arch Linux repo Python) venv with just pytest 3.6.3 installed.

Seems related to #2775.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported collection with conftest.py and test_foo.py using pytest --pyargs from quux, its parent, and an unrelated directory. Compare the nodeids for pkg, pkg.tests, and pkg.tests.test_foo, then trace pytest's collection and nodeid handling. Done means all three invocations consistently report pkg/tests/test_foo.py::test_bar.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.