pytest-dev / pytest-dev/pytest
Run pexpect tests on CI by default
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
pytestertests that requirepexpectare silently skipped ifpexpectis unavailable:
- We default to not installing
pexpecteven on CI, only having it as an optional dependency with a-pexpectfactor:
https://github.com/pytest-dev/pytest/blob/f373974707f57a0b28d12563e4d03c7cd54c70d9/tox.ini#L79
- The only CI environment where we include that is a Python 3.8 one:
-
Yet, we have 40 (!) selftests that use
pexpect- most notably, most of test_debugging.py (38 of 59 skipped) -
And that caused us to miss #12497 on our own CI
Why is pexpect an optional factor at all? From what I can gather, that was introduced in bd8a2cc18c1898673665023556e73134e78e4d75 because "it doesn't install on Windows anymore" back in 2013.
Nowadays, it seems to be partially available for Windows, though not pexpect.spawn which we use for Pytester.spawn():
I think we should either:
- Figure out if we can change
Pytesterin a backwards-compatible way to use the cross-platformPopenSpawninstead. Pexpect claims:
PopenSpawnis not a direct replacement forspawn. Many programs only offer interactive behaviour if they detect that they are running in a terminal. When run byPopenSpawn, they may behave differently.
- Or if not, at least install
pexpectunconditionally on Linux (and perhaps macOS) as part of our dev dependencies, using a environment marker (added in pip 6.0 in 2014, so that wasn't an option back when the tox factor was introduced).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read tox.ini and .github/workflows/test.yml to understand the current pexpect factor and CI matrix, then inspect src/_pytest/pytester.py and testing/test_debugging.py for the skipped cases. Determine whether the existing spawn behavior can be supported cross-platform or whether pexpect should be installed on supported CI systems; done means the relevant tests run by default without inappropriate skips.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100