pytest-dev / pytest-dev/pytest
pytest_addoption is called after CLI argparsing when conftest is in a subdirectory of `tests` and the path is not specified
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
I have hit a rather confusing edge case when using pytest_addoption in a conftest.py that resides in a sub-directory of the tests directory. It appears that pytest is then performing the CLI parsing before collecting the tests and the conftest.py (which leads to really weird issues). On the other hand, when I specify the test path manually, then pytest will do "the right thing" (= perform the CLI parsing after readin in conftest from the directory).
This can be reproduced as follows:
$ poetry init --name "test" --python "^3.9" --description "Just a test" --author "me" -n --dev-dependency="pytest"
$ poetry install
$ mkdir -p tests/base/
$ touch tests/__init__.py
$ cat <<EOF > tests/base/conftest.py
def pytest_addoption(parser):
parser.addoption("--foobar")
EOF
$ cat <<EOF > tests/base/test_something.py
def pytest_addoption(parser):
parser.addoption("--foobar")
EOF
Now, I would intuitively expect that I could now run poetry run pytest --foobar but this does not work:
$ poetry run pytest --foobar
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --foobar
inifile: None
rootdir: /tmp/pytest-test
However, if I explicitly set the path where to look for tests, then it works:
$ poetry run pytest --foobar tests/base/*py
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /tmp/pytest-test
plugins: cov-3.0.0
collected 1 item
tests/base/test_something.py . [100%]
============================================================================================================= 1 passed in 0.01s =============================================================================================================
Please also note that pytest is still executing conftest.py if I don't specify a CLI flag. One can for example modify pytest_addoption(parser) as follows:
def pytest_addoption(parser):
raise ValueError("foo")
And executing poetry run pytest leads to the expected error:
$ poetry run pytest
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /tmp/pytest-test
plugins: cov-3.0.0
collected 0 items / 1 error
================================================================================================================== ERRORS ===================================================================================================================
_______________________________________________________________________________________________________ ERROR collecting test session _______________________________________________________________________________________________________
/home/dan/.cache/pypoetry/virtualenvs/test-HwDUSR58-py3.9/lib/python3.9/site-packages/pluggy/_manager.py:115: in register
hook._maybe_apply_history(hookimpl)
/home/dan/.cache/pypoetry/virtualenvs/test-HwDUSR58-py3.9/lib/python3.9/site-packages/pluggy/_hooks.py:300: in _maybe_apply_history
res = self._hookexec(self.name, [method], kwargs, False)
/home/dan/.cache/pypoetry/virtualenvs/test-HwDUSR58-py3.9/lib/python3.9/site-packages/pluggy/_manager.py:80: in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
tests/base/conftest.py:3: in pytest_addoption
raise ValueError("foo")
E ValueError: foo
========================================================================================================== short test summary info ==========================================================================================================
ERROR - ValueError: foo
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================= 1 error in 0.07s ==============================================================================================================
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproduction in tests/base/conftest.py and tests/base/test_something.py, then run the two pytest commands shown in the issue to compare argument parsing with and without an explicit path. Trace when pytest_addoption is registered relative to CLI parsing. Done means --foobar is accepted when tests are discovered automatically, while the existing explicit-path behavior remains working.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100