pytest-dev / pytest-dev/pytest
Take --pyargs into account to find initial conftests (was: py.test --pyargs my_pkg doesn't run pytest_addoption in my_pkg/conftest.py)
Open
Nobody has claimed this yet.
type: enhancement
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
As part of our build process, we run the test suite after creating the packaging to check that the installed package works as intended. We recently added an option to py.test for some conditional testing, by placing a function pytest_addoption in the conftest.py file that is in the root of the package (so next to my_pkg/__init__.py). When calling py.test on the source, this works as intended. Tests are skipped when we omit the option, and they are executed when we pass the option.
my_pkg/conftest.py:
def pytest_addoption(parser):
"""Adds options to the pytest parser."""
parser.addoption("--azure-storage-emulator",
action="store_true",
help="run tests that depend on the azure storage emulator")
However, when we run py.test --pyargs my_pkg in an environment where the built package is installed, we see the following error message (even though the test suite is run):
=================================== ERRORS ====================================
______________________________ ERROR collecting ______________________________
..\..\envs\_test\lib\site-packages\my_pkg\storage\tests\test_azure.py:18: in <module>
not pytest.config.getoption("--azure-storage-emulator"),
..\..\envs\_test\lib\site-packages\_pytest\config.py:1061: in getoption
raise ValueError("no option named %r" % (name,))
E ValueError: no option named 'azure_storage_emulator'
All other tests, including those depending on fixtures in the same conftest.py file, simply pass.
We're running on Windows and Linux, and both on dev machines as on CI (AppVeyor and Codeship), this scenario occurs. We're using the following related packages:
pytest: 2.9.1-py_1
pytest-cov: 2.2.1-py35_0
pytest-mock: 0.10.1-py35_0
pytest-xdist: 1.14-py35_0
python: 3.5.1-4
Running with --traceconfig does show that the conftest.py file is registered:
PLUGIN registered: <module 'my_pkg.conftest' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\my_pkg\\conftest.py'>
full output:
[_test] C:\Users\Korijn\Miniconda3\conda-bld\test-tmp_dir>py.test --traceconfig --pyargs my_pkg
PLUGIN registered: <_pytest.config.PytestPluginManager object at 0x0000027EA83061D0>
PLUGIN registered: <_pytest.config.Config object at 0x0000027EA8999198>
PLUGIN registered: <module '_pytest.mark' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\mark.py'>
PLUGIN registered: <module '_pytest.main' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\main.py'>
PLUGIN registered: <module '_pytest.terminal' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\terminal.py'>
PLUGIN registered: <module '_pytest.runner' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\runner.py'>
PLUGIN registered: <module '_pytest.python' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\python.py'>
PLUGIN registered: <module '_pytest.pdb' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\pdb.py'>
PLUGIN registered: <module '_pytest.unittest' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\unittest.py'>
PLUGIN registered: <module '_pytest.capture' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\capture.py'>
PLUGIN registered: <module '_pytest.skipping' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\skipping.py'>
PLUGIN registered: <module '_pytest.tmpdir' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\tmpdir.py'>
PLUGIN registered: <module '_pytest.monkeypatch' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\monkeypatch.py'>
PLUGIN registered: <module '_pytest.recwarn' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\recwarn.py'>
PLUGIN registered: <module '_pytest.pastebin' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\pastebin.py'>
PLUGIN registered: <module '_pytest.helpconfig' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\helpconfig.py'>
PLUGIN registered: <module '_pytest.nose' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\nose.py'>
PLUGIN registered: <module '_pytest.assertion' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\assertion\\__init__.py'>
PLUGIN registered: <module '_pytest.genscript' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\genscript.py'>
PLUGIN registered: <module '_pytest.junitxml' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\junitxml.py'>
PLUGIN registered: <module '_pytest.resultlog' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\resultlog.py'>
PLUGIN registered: <module '_pytest.doctest' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\doctest.py'>
PLUGIN registered: <module '_pytest.cacheprovider' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\_pytest\\cacheprovider.py'>
PLUGIN registered: <module 'pytest_cov.plugin' from 'c:\\users\\korijn\\miniconda3\\envs\\_test\\lib\\site-packages\\pytest_cov-2.2.1-py3.5.egg\\pytest_cov\\plugin.py'>
PLUGIN registered: <module 'pytest_mock' from 'c:\\users\\korijn\\miniconda3\\envs\\_test\\lib\\site-packages\\pytest_mock-0.10.1-py3.5.egg\\pytest_mock.py'>
PLUGIN registered: <module 'xdist.plugin' from 'c:\\users\\korijn\\miniconda3\\envs\\_test\\lib\\site-packages\\pytest_xdist-1.14-py3.5.egg\\xdist\\plugin.py'>
PLUGIN registered: <module 'xdist.looponfail' from 'c:\\users\\korijn\\miniconda3\\envs\\_test\\lib\\site-packages\\pytest_xdist-1.14-py3.5.egg\\xdist\\looponfail.py'>
PLUGIN registered: <module 'xdist.boxed' from 'c:\\users\\korijn\\miniconda3\\envs\\_test\\lib\\site-packages\\pytest_xdist-1.14-py3.5.egg\\xdist\\boxed.py'>
PLUGIN registered: <_pytest.capture.CaptureManager object at 0x0000027EA95BF208>
PLUGIN registered: <Session 'test-tmp_dir'>
PLUGIN registered: <_pytest.cacheprovider.LFPlugin object at 0x0000027EA95C9E80>
PLUGIN registered: <_pytest.terminal.TerminalReporter object at 0x0000027EA95F6320>
PLUGIN registered: <_pytest.python.FixtureManager object at 0x0000027EA960E7F0>
============================= test session starts =============================
platform win32 -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
using: pytest-2.9.1 pylib-1.4.31
setuptools registered plugins:
pytest-cov-2.2.1 at c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_cov-2.2.1-py3.5.egg\pytest_cov\plugin.py
pytest-mock-0.10.1 at c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_mock-0.10.1-py3.5.egg\pytest_mock.py
pytest-xdist-1.14 at c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_xdist-1.14-py3.5.egg\xdist\plugin.py
pytest-xdist-1.14 at c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_xdist-1.14-py3.5.egg\xdist\looponfail.py
pytest-xdist-1.14 at c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_xdist-1.14-py3.5.egg\xdist\boxed.py
active plugins:
main : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\main.py
mark : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\mark.py
pytest_mock : c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_mock-0.10.1-py3.5.egg\pytest_mock.py
python : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\python.py
genscript : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\genscript.py
runner : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\runner.py
pytest_cov : c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_cov-2.2.1-py3.5.egg\pytest_cov\plugin.py
terminal : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\terminal.py
skipping : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\skipping.py
tmpdir : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\tmpdir.py
resultlog : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\resultlog.py
lfplugin : <_pytest.cacheprovider.LFPlugin object at 0x0000027EA95C9E80>
xdist.boxed : c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_xdist-1.14-py3.5.egg\xdist\boxed.py
assertion : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\assertion\__init__.py
2743010877904 : <_pytest.config.PytestPluginManager object at 0x0000027EA83061D0>
nose : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\nose.py
helpconfig : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\helpconfig.py
pdb : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\pdb.py
cacheprovider : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\cacheprovider.py
pytestconfig : <_pytest.config.Config object at 0x0000027EA8999198>
pastebin : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\pastebin.py
doctest : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\doctest.py
junitxml : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\junitxml.py
funcmanage : <_pytest.python.FixtureManager object at 0x0000027EA960E7F0>
recwarn : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\recwarn.py
session : <Session 'test-tmp_dir'>
xdist : c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_xdist-1.14-py3.5.egg\xdist\plugin.py
terminalreporter : <_pytest.terminal.TerminalReporter object at 0x0000027EA95F6320>
capturemanager : <_pytest.capture.CaptureManager object at 0x0000027EA95BF208>
xdist.looponfail : c:\users\korijn\miniconda3\envs\_test\lib\site-packages\pytest_xdist-1.14-py3.5.egg\xdist\looponfail.py
capture : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\capture.py
monkeypatch : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\monkeypatch.py
unittest : C:\Users\Korijn\Miniconda3\envs\_test\lib\site-packages\_pytest\unittest.py
rootdir: C:\Users\Korijn\Miniconda3\conda-bld\test-tmp_dir, inifile:
plugins: cov-2.2.1, mock-0.10.1, xdist-1.14
PLUGIN registered: <module 'my_pkg.conftest' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\my_pkg\\conftest.py'>
PLUGIN registered: <module 'my_pkg.sub_pkg.conftest' from 'C:\\Users\\Korijn\\Miniconda3\\envs\\_test\\lib\\site-packages\\my_pkg\\sub_pkg\\conftest.py'>
collected 352 items / 1 errors
. ................................................................................................................................................................................................................................................................................................................................................................
=================================== ERRORS ====================================
______________________________ ERROR collecting ______________________________
..\..\envs\_test\lib\site-packages\my_pkg\storage\tests\test_azure.py:18: in <module>
not pytest.config.getoption("--azure-storage-emulator"),
..\..\envs\_test\lib\site-packages\_pytest\config.py:1061: in getoption
raise ValueError("no option named %r" % (name,))
E ValueError: no option named 'azure_storage_emulator'
==================== 352 passed, 1 error in 57.56 seconds =====================
I tried adding a default value:
azure_storage_emulator = pytest.mark.skipif(
not pytest.config.getoption("--azure-storage-emulator", default=False),
reason="need --azure-storage-emulator option to run"
)
This avoids the exception, but I still can't pass the option:
[_test] C:\Users\Korijn\Miniconda3\conda-bld\test-tmp_dir>py.test --pyargs my_pkg
============================= test session starts =============================
platform win32 -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: C:\Users\Korijn\Miniconda3\conda-bld\test-tmp_dir, inifile:
plugins: cov-2.2.1, mock-0.10.1, xdist-1.14
collected 363 items
. ...........................................................................................................................................................................................................................................
........................................................................................................s.......................
=================== 362 passed, 1 skipped in 60.17 seconds ====================
[_test] C:\Users\Korijn\Miniconda3\conda-bld\test-tmp_dir>py.test --pyargs my_pkg --azure-storage-emulator
usage: py.test-script.py [options] [file_or_dir] [file_or_dir] [...]
py.test-script.py: error: unrecognized arguments: --azure-storage-emulator
inifile: None
rootdir: C:\Users\Korijn\Miniconda3\conda-bld\test-tmp_dir
[_test] C:\Users\Korijn\Miniconda3\conda-bld\test-tmp_dir>
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with --pyargs handling and initial conftest discovery, using my_pkg/conftest.py and the failing storage/tests/test_azure.py scenario as the reproduction. Run py.test --pyargs my_pkg with --traceconfig; done means pytest_addoption is available before collection and test_azure.py no longer raises the missing-option error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100