pytest-dev / pytest-dev/pytest

Re-evaluation of commandline after(!) full import of all conftest.py files

Open
#9,739 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: deprecation type: docs
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

In some cases, a pytest_addoption() in conftest.py is not evaluated before the commandline is executed.
This seems to be the case if the conftest.py resides in an installed package (including tests) and you want to execute the test using
pytest --pyargs some_package --myoption

If possible, I would propose to ignore invalid options in a first run, until actually all relevant conftest.py have been imported and then re-evalute the invalid options.

Example:

The test is supposed to support a customized test option I added via pytest_addoption() in a conftest.py that I placed within the package:

folder tree:

$HOME/install/pytest_testing
$HOME/install/pytest_testing/__init__.py           # empty
$HOME/install/pytest_testing/tests/__init__.py  # empty
$HOME/install/pytest_testing/tests/conftest.py # contains pytest_addoption()
$HOME/install/pytest_testing/tests/test_package.py # contains a test_package()

Now, I am trying to run the test from somewhere, so not within the actual package folder tree (source code or installation) via

/somewhere > export PYTHONPATH=$HOME/install
/somewhere > pytest --pyargs pytest_testing -s

==========================  session starts ========================================
platform linux -- Python 3.8.12, pytest-7.0.1, pluggy-1.0.0
rootdir: /somewhere
collecting ... 
-----------------------------------------------------------
$HOME/install/pytest_testing/tests/conftest.py:
pytest_addoption:  option added
-----------------------------------------------------------
-----------------------------------------------------------
collected 1 item                                                                                         
-----------------------------------------------------------
test_package: myoption = False
-----------------------------------------------------------
======================= 1 passed in 0.03s ==========================================

BUT

/somewhere > pytest --pyargs pytest_testing -s --myoption

ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --myoption
  inifile: None
  rootdir: /somewhere

So, in the second case --myoption is evaluated before conftest.py has been. However the first call clearly shows that the option is correctly added, however too late.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the two documented commands with the installed package layout and compare command-line parsing with loading tests/conftest.py. Start at pytest's --pyargs handling and conftest.py loading path; done means an option added by pytest_addoption() in the package's conftest.py is accepted when supplied on the initial command line.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.