pytest-dev / pytest-dev/pytest

pytest_addoption is called after CLI argparsing when conftest is in a subdirectory of `tests` and the path is not specified

Open
#9,337 0 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: config topic: reporting type: question
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

I have hit a rather confusing edge case when using pytest_addoption in a conftest.py that resides in a sub-directory of the tests directory. It appears that pytest is then performing the CLI parsing before collecting the tests and the conftest.py (which leads to really weird issues). On the other hand, when I specify the test path manually, then pytest will do "the right thing" (= perform the CLI parsing after readin in conftest from the directory).

This can be reproduced as follows:

$ poetry init --name "test" --python "^3.9" --description "Just a test" --author "me" -n --dev-dependency="pytest"
$ poetry install
$ mkdir -p tests/base/
$ touch tests/__init__.py
$ cat <<EOF > tests/base/conftest.py
def pytest_addoption(parser):
    parser.addoption("--foobar")
EOF
$ cat <<EOF > tests/base/test_something.py
def pytest_addoption(parser):
    parser.addoption("--foobar")
EOF

Now, I would intuitively expect that I could now run poetry run pytest --foobar but this does not work:

$ poetry run pytest --foobar 
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --foobar
  inifile: None
  rootdir: /tmp/pytest-test

However, if I explicitly set the path where to look for tests, then it works:

$ poetry run pytest --foobar tests/base/*py
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /tmp/pytest-test
plugins: cov-3.0.0
collected 1 item                                                                                                                                                                                                                            

tests/base/test_something.py .                                                                                                                                                                                                        [100%]

============================================================================================================= 1 passed in 0.01s =============================================================================================================

Please also note that pytest is still executing conftest.py if I don't specify a CLI flag. One can for example modify pytest_addoption(parser) as follows:

def pytest_addoption(parser):
    raise ValueError("foo")

And executing poetry run pytest leads to the expected error:

$ poetry run pytest 
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /tmp/pytest-test
plugins: cov-3.0.0
collected 0 items / 1 error                                                                                                                                                                                                                 

================================================================================================================== ERRORS ===================================================================================================================
_______________________________________________________________________________________________________ ERROR collecting test session _______________________________________________________________________________________________________
/home/dan/.cache/pypoetry/virtualenvs/test-HwDUSR58-py3.9/lib/python3.9/site-packages/pluggy/_manager.py:115: in register
    hook._maybe_apply_history(hookimpl)
/home/dan/.cache/pypoetry/virtualenvs/test-HwDUSR58-py3.9/lib/python3.9/site-packages/pluggy/_hooks.py:300: in _maybe_apply_history
    res = self._hookexec(self.name, [method], kwargs, False)
/home/dan/.cache/pypoetry/virtualenvs/test-HwDUSR58-py3.9/lib/python3.9/site-packages/pluggy/_manager.py:80: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
tests/base/conftest.py:3: in pytest_addoption
    raise ValueError("foo")
E   ValueError: foo
========================================================================================================== short test summary info ==========================================================================================================
ERROR  - ValueError: foo
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================= 1 error in 0.07s ==============================================================================================================

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproduction in tests/base/conftest.py and tests/base/test_something.py, then run the two pytest commands shown in the issue to compare argument parsing with and without an explicit path. Trace when pytest_addoption is registered relative to CLI parsing. Done means --foobar is accepted when tests are discovered automatically, while the existing explicit-path behavior remains working.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.