pytest-dev / pytest-dev/pytest

-p silently fails if parameter is an invalid plugin

Open
#14,135 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

  • a detailed description of the bug or problem you are having

    When using --disable-plugin-autoload, some plugins, eg pytest-recording, are not picked unless the parameter to -p is the entry point, not the module name.
    However, this fails silently -- using the module name doesn't cause any warnings from pytest.

    In particular, this can be detected by checking if pytest accepts the command line flags defined by these plugins -- with -p pytest_recording, --block_network isn't accepted, but with -p recording it is.

  • output of pip list from the virtual environment you are using

Package          Version
---------------- -------
iniconfig        2.3.0
packaging        25.0
pip              25.3
pluggy           1.6.0
Pygments         2.19.2
pytest           9.0.2
pytest-recording 0.13.4
PyYAML           6.0.3
vcrpy            8.1.1
wrapt            2.0.1
  • pytest and operating system versions: pytest-9.0.2, Arch Linux

  • minimal example if possible

    With

`pytest -p pytest_recording` ``` $ pytest --disable-plugin-autoload -p pytest_recording --trace-config PLUGIN registered: <_pytest.config.PytestPluginManager object at 0x7f5230ca6660> PLUGIN registered: <_pytest.config.Config object at 0x7f52309101a0> PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: > err=> in_=> _state='suspended' _in_suspended=False> _capture_fixture=None> PLUGIN registered: testsfailed=0 testscollected=0> PLUGIN registered: <_pytest.cacheprovider.LFPlugin object at 0x7f5230978ec0> PLUGIN registered: <_pytest.cacheprovider.NFPlugin object at 0x7f5230979940> PLUGIN registered: PLUGIN registered: <_pytest.terminal.TerminalReporter object at 0x7f522fd6c1a0> PLUGIN registered: <_pytest.logging.LoggingPlugin object at 0x7f522fd6c590> PLUGIN registered: <_pytest.fixtures.FixtureManager object at 0x7f522fd6cec0> =============================================================================================== test session starts ================================================================================================ platform linux -- Python 3.14.2, pytest-9.0.2, pluggy-1.6.0 using: pytest-9.0.2 active plugins: 139990982616672 : <_pytest.config.PytestPluginManager object at 0x7f5230ca6660> pytestconfig : <_pytest.config.Config object at 0x7f52309101a0> mark : /tmp/venv/lib/python3.14/site-packages/_pytest/mark/__init__.py main : /tmp/venv/lib/python3.14/site-packages/_pytest/main.py runner : /tmp/venv/lib/python3.14/site-packages/_pytest/runner.py fixtures : /tmp/venv/lib/python3.14/site-packages/_pytest/fixtures.py helpconfig : /tmp/venv/lib/python3.14/site-packages/_pytest/helpconfig.py python : /tmp/venv/lib/python3.14/site-packages/_pytest/python.py terminal : /tmp/venv/lib/python3.14/site-packages/_pytest/terminal.py debugging : /tmp/venv/lib/python3.14/site-packages/_pytest/debugging.py unittest : /tmp/venv/lib/python3.14/site-packages/_pytest/unittest.py capture : /tmp/venv/lib/python3.14/site-packages/_pytest/capture.py skipping : /tmp/venv/lib/python3.14/site-packages/_pytest/skipping.py legacypath : /tmp/venv/lib/python3.14/site-packages/_pytest/legacypath.py tmpdir : /tmp/venv/lib/python3.14/site-packages/_pytest/tmpdir.py monkeypatch : /tmp/venv/lib/python3.14/site-packages/_pytest/monkeypatch.py recwarn : /tmp/venv/lib/python3.14/site-packages/_pytest/recwarn.py pastebin : /tmp/venv/lib/python3.14/site-packages/_pytest/pastebin.py assertion : /tmp/venv/lib/python3.14/site-packages/_pytest/assertion/__init__.py junitxml : /tmp/venv/lib/python3.14/site-packages/_pytest/junitxml.py doctest : /tmp/venv/lib/python3.14/site-packages/_pytest/doctest.py cacheprovider : /tmp/venv/lib/python3.14/site-packages/_pytest/cacheprovider.py setuponly : /tmp/venv/lib/python3.14/site-packages/_pytest/setuponly.py setupplan : /tmp/venv/lib/python3.14/site-packages/_pytest/setupplan.py stepwise : /tmp/venv/lib/python3.14/site-packages/_pytest/stepwise.py unraisableexception : /tmp/venv/lib/python3.14/site-packages/_pytest/unraisableexception.py threadexception : /tmp/venv/lib/python3.14/site-packages/_pytest/threadexception.py warnings : /tmp/venv/lib/python3.14/site-packages/_pytest/warnings.py logging : /tmp/venv/lib/python3.14/site-packages/_pytest/logging.py reports : /tmp/venv/lib/python3.14/site-packages/_pytest/reports.py faulthandler : /tmp/venv/lib/python3.14/site-packages/_pytest/faulthandler.py subtests : /tmp/venv/lib/python3.14/site-packages/_pytest/subtests.py pytest_recording : /tmp/venv/lib/python3.14/site-packages/pytest_recording/__init__.py capturemanager : > err=> in_=> _state='suspended' _in_suspended=False> _capture_fixture=None> session : testsfailed=0 testscollected=0> lfplugin : <_pytest.cacheprovider.LFPlugin object at 0x7f5230978ec0> nfplugin : <_pytest.cacheprovider.NFPlugin object at 0x7f5230979940> legacypath-tmpdir : terminalreporter : <_pytest.terminal.TerminalReporter object at 0x7f522fd6c1a0> logging-plugin : <_pytest.logging.LoggingPlugin object at 0x7f522fd6c590> funcmanage : <_pytest.fixtures.FixtureManager object at 0x7f522fd6cec0> rootdir: /tmp ```
`pytest -p recording` ``` $ pytest --disable-plugin-autoload -p recording --block-network --trace-config PLUGIN registered: <_pytest.config.PytestPluginManager object at 0x7f5230ca6660> PLUGIN registered: <_pytest.config.Config object at 0x7f52309101a0> PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: PLUGIN registered: > err=> in_=> _state='suspended' _in_suspended=False> _capture_fixture=None> PLUGIN registered: testsfailed=0 testscollected=0> PLUGIN registered: <_pytest.cacheprovider.LFPlugin object at 0x7f5230978ec0> PLUGIN registered: <_pytest.cacheprovider.NFPlugin object at 0x7f5230979940> PLUGIN registered: PLUGIN registered: <_pytest.terminal.TerminalReporter object at 0x7f522fd6c1a0> PLUGIN registered: <_pytest.logging.LoggingPlugin object at 0x7f522fd6c590> PLUGIN registered: <_pytest.fixtures.FixtureManager object at 0x7f522fd6cec0> =============================================================================================== test session starts ================================================================================================ platform linux -- Python 3.14.2, pytest-9.0.2, pluggy-1.6.0 using: pytest-9.0.2 active plugins: 139990982616672 : <_pytest.config.PytestPluginManager object at 0x7f5230ca6660> pytestconfig : <_pytest.config.Config object at 0x7f52309101a0> mark : /tmp/venv/lib/python3.14/site-packages/_pytest/mark/__init__.py main : /tmp/venv/lib/python3.14/site-packages/_pytest/main.py runner : /tmp/venv/lib/python3.14/site-packages/_pytest/runner.py fixtures : /tmp/venv/lib/python3.14/site-packages/_pytest/fixtures.py helpconfig : /tmp/venv/lib/python3.14/site-packages/_pytest/helpconfig.py python : /tmp/venv/lib/python3.14/site-packages/_pytest/python.py terminal : /tmp/venv/lib/python3.14/site-packages/_pytest/terminal.py debugging : /tmp/venv/lib/python3.14/site-packages/_pytest/debugging.py unittest : /tmp/venv/lib/python3.14/site-packages/_pytest/unittest.py capture : /tmp/venv/lib/python3.14/site-packages/_pytest/capture.py skipping : /tmp/venv/lib/python3.14/site-packages/_pytest/skipping.py legacypath : /tmp/venv/lib/python3.14/site-packages/_pytest/legacypath.py tmpdir : /tmp/venv/lib/python3.14/site-packages/_pytest/tmpdir.py monkeypatch : /tmp/venv/lib/python3.14/site-packages/_pytest/monkeypatch.py recwarn : /tmp/venv/lib/python3.14/site-packages/_pytest/recwarn.py pastebin : /tmp/venv/lib/python3.14/site-packages/_pytest/pastebin.py assertion : /tmp/venv/lib/python3.14/site-packages/_pytest/assertion/__init__.py junitxml : /tmp/venv/lib/python3.14/site-packages/_pytest/junitxml.py doctest : /tmp/venv/lib/python3.14/site-packages/_pytest/doctest.py cacheprovider : /tmp/venv/lib/python3.14/site-packages/_pytest/cacheprovider.py setuponly : /tmp/venv/lib/python3.14/site-packages/_pytest/setuponly.py setupplan : /tmp/venv/lib/python3.14/site-packages/_pytest/setupplan.py stepwise : /tmp/venv/lib/python3.14/site-packages/_pytest/stepwise.py unraisableexception : /tmp/venv/lib/python3.14/site-packages/_pytest/unraisableexception.py threadexception : /tmp/venv/lib/python3.14/site-packages/_pytest/threadexception.py warnings : /tmp/venv/lib/python3.14/site-packages/_pytest/warnings.py logging : /tmp/venv/lib/python3.14/site-packages/_pytest/logging.py reports : /tmp/venv/lib/python3.14/site-packages/_pytest/reports.py faulthandler : /tmp/venv/lib/python3.14/site-packages/_pytest/faulthandler.py subtests : /tmp/venv/lib/python3.14/site-packages/_pytest/subtests.py pytest_recording : /tmp/venv/lib/python3.14/site-packages/pytest_recording/__init__.py capturemanager : > err=> in_=> _state='suspended' _in_suspended=False> _capture_fixture=None> session : testsfailed=0 testscollected=0> lfplugin : <_pytest.cacheprovider.LFPlugin object at 0x7f5230978ec0> nfplugin : <_pytest.cacheprovider.NFPlugin object at 0x7f5230979940> legacypath-tmpdir : terminalreporter : <_pytest.terminal.TerminalReporter object at 0x7f522fd6c1a0> logging-plugin : <_pytest.logging.LoggingPlugin object at 0x7f522fd6c590> funcmanage : <_pytest.fixtures.FixtureManager object at 0x7f522fd6cec0> rootdir: /tmp ```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with pytest --disable-plugin-autoload -p pytest_recording and -p recording, comparing plugin registration and command-line option handling. Trace pytest's plugin-loading entry point and related tests for -p parameters. Done means an invalid plugin parameter is no longer silently accepted and the behavior is covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.