pytest-dev / pytest-dev/pytest

How to disable plugin based on environment?

Open
#7,675 15 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: config type: proposal
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

In python/typed_ast#144, I confirmed that typed_ast, and by extension pytest-black and pytest-mypy, are not supported on PyPy (and cause crashes on installation). I'd like to fashion a technique that reflects this limitation and bypasses the installation and enabling of these plugins when testing on PyPy.

I've found I can selectively include or exclude the plugin modules by adding environment markers to the requirement:

	pytest-black >= 0.3.7; platform_python_implementation != "PyPy"
	pytest-cov
	pytest-mypy; platform_python_implementation != "PyPy"

However, these plugins require a two-step installation process:

  1. Install the dependency.
  2. Enable the plugin with --black or --mypy during pytest invocation.

Leaving the --<plugin> in pytest.ini (example) while deselecting the plugin for installation results in an InvocationError:

keyring master $ tox -r -e pypy3
pypy3 recreate: /Users/jaraco/code/main/keyring/.tox/pypy3
pypy3 develop-inst: /Users/jaraco/code/main/keyring
pypy3 installed: attrs==20.1.0,cffi==1.14.0,coverage==5.2.1,docutils==0.16,flake8==3.8.3,greenlet==0.4.13,importlib-metadata==1.7.0,iniconfig==1.0.1,-e git+gh://jaraco/keyring@a6980e4b0a0c25143ab027e437a1b5b81f9c0bf5#egg=keyring,mccabe==0.6.1,more-itertools==8.4.0,packaging==20.4,pluggy==0.13.1,py==1.9.0,pycodestyle==2.6.0,pyflakes==2.2.0,pyparsing==2.4.7,pytest==6.0.1,pytest-checkdocs==2.1.1,pytest-cov==2.10.1,pytest-flake8==1.0.6,readline==6.2.4.1,six==1.15.0,toml==0.10.1,zipp==3.1.0
pypy3 run-test-pre: PYTHONHASHSEED='4203199892'
pypy3 run-test: commands[0] | pytest
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --black --mypy
  inifile: /Users/jaraco/code/main/keyring/pytest.ini
  rootdir: /Users/jaraco/code/main/keyring

ERROR: InvocationError for command /Users/jaraco/code/main/keyring/.tox/pypy3/bin/pytest (exited with code 4)
_____________________________________________________________ summary ______________________________________________________________
ERROR:   pypy3: commands failed

Best I can tell, there's no easy way to programmatically disable the plugin (or only enable it if present).

In pytest-dev/pytest-cov#418, I encountered a similar problem where it was suitable to install coverage but its presence needed to be disabled at runtime. The workaround for that approach was specifically tuned to implementation details of that plugin.

What I'd really like is a uniform way for plugins to declare there presence, for them to be enabled by default, and layers of configuration (global, system, user, environment, invocation) where the enabling of that feature can be toggled on or off based on certain factors (maybe environment markers or maybe arbitrary Python logic).

But thinking short-term, are there any options available to selectively disable the enabling of the black and mypy plugins, such that they're enabled by default, but disabled when the environment is built on PyPy?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the pytest.ini plugin options and the tox pypy3 test invocation described in the report. Reproduce the unrecognized --black and --mypy arguments on PyPy, then investigate how plugin enabling and configuration are handled. Done means a documented, uniform way to keep these plugins enabled by default while disabling them on PyPy without an InvocationError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.