astropy / astropy/pytest-filter-subpackage

BUG: -P nonexistentmodule should fail

Open
#9 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
5
Forks
7
PR merge metrics
No merged PRs in 30d

Description

`python setup.py test -P nonexistent` used to raise a Valuerror:

```
python setup.py test -P nonexistent/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
warnings.warn(
running test
INFO: installing to temporary directory: /private/var/folders/9s/070g0pd502q70k3gffpxv8km0000gq/T/astroquery-test-_jo10zsr [astropy.tests.command]
no previously-included directories found matching 'build'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'docs/api'
no previously-included directories found matching 'astropy_helpers/build'
warning: no previously-included files matching '*.o' found anywhere in distribution
Traceback (most recent call last):
File "", line 1, in
File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 259, in test
return runner.run_tests(**kwargs)
File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 604, in run_tests
return super().run_tests(**kwargs)
File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 216, in run_tests
args = self._generate_args(**kwargs)
File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 145, in _generate_args
result = func(keywords[keyword], keywords)
File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 41, in keyword
return f(*args, **kwargs)
File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 351, in package
self.package_path = self.packages_path(package, self.base_path,
File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 316, in packages_path
raise ValueError(error.format(**info))
ValueError: package to test is not found: nonexistent (at path /private/var/folders/9s/070g0pd502q70k3gffpxv8km0000gq/T/astroquery-test-_jo10zsr/lib/python3.9/site-packages/astroquery/nonexistent).

```

but unfortunately it's not the case anymore with pytest:

```
pytest -P nonexistent
================================================= test session starts ==================================================
platform darwin -- Python 3.9.1, pytest-7.1.2, pluggy-0.13.1
WARNING: AstropyDeprecationWarning: The ignore_warnings class is deprecated and may be removed in a future version.
Use pytest.mark.filterwarnings instead. [pytest_astropy_header.display]
WARNING: AstropyDeprecationWarning: The catch_warnings class is deprecated and may be removed in a future version.
Use pytest.warns instead. [astropy.tests.helper]

Running tests with astroquery version 0.4.7.dev8203_testrun.
Running tests in astroquery docs.

Date: 2022-11-09T13:29:18

Platform: macOS-10.16-x86_64-i386-64bit

Executable: /Users/bsipocz/.pyenv/versions/3.9.1/bin/python3.9

Full Python Version:
3.9.1 (default, Jan 11 2021, 15:50:22)
[Clang 10.0.0 (clang-1000.11.45.5)]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Package versions:
Numpy: 1.23.4
Scipy: 1.5.4
Matplotlib: 3.5.3
h5py: 3.1.0
Pandas: 1.4.4
Astropy: 5.1.1
regions: 0.6
pyVO: 1.5.dev12+gbacb8ab
mocpy: 0.10.1
astropy-healpix: 0.7
vamdclib: not available
astropy-helpers: 4.0.1

Using Astropy options: remote_data: none.

rootdir: /Users/bsipocz/munka/devel/worktrees/astroquery/testings, configfile: setup.cfg, testpaths: astroquery, docs
plugins: anyio-3.6.1, arraydiff-0.3, requests-mock-1.9.3, rerunfailures-10.2, mock-3.7.0, nbval-0.9.6, dependency-0.5.1, astropy-0.10.0, hypothesis-6.3.1, filter-subpackage-0.1.1, openfiles-0.5.0, doctestplus-0.11.1, astropy-header-0.1.2, cov-3.0.0, remotedata-0.3.4.dev11+g319a61b
collected 0 items

================================================ no tests ran in 1.70s =================================================
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the issue with `pytest -P nonexistent` and compare its zero-test result with `python setup.py test -P nonexistent`, which raised a ValueError. The payload names no source file or test, so locate the plugin's handling of the `-P` option and add coverage showing that a nonexistent package fails rather than collecting zero tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.