Flake8 problems
- Dominant language
- Python
- Stars
- 226
- Forks
- 56
- PR merge metrics
- No merged PRs in 30d
Description
There are two problems:
1. I don't think pytest is running flake8 like it should be, and
2. When you do force flake8 to run with pytest manually, there are failures. Something is wrong with the pytest flake8 hook (maybe a version mismatch between pytest, flake8, and pytest-flake8?)
**Describe the issue**:
We have `--flake8` as a pytest `addopts` extra:
```
[tool:pytest]
addopts = --flake8
```
...which I think is equivalent to the command line call
```
pytest --flake8
```
But when I run just `pytest` from the command line, it doesn't seem to pick up the extra addopts bit. All the tests pass with just `pytest`, but there are failures when I run `pytest --flake8` explicitly.
**Minimal Complete Verifiable Example**:
1. Install a development version of dask-image, including the test requirements. Also pip install pandas, as that is a test requirement for the new-ish find_objects function.
3. Run `pytest`, and observe the tests passing.
4. Run `pytest --flake8` (which should be equivalent to the above, given we are using addopts "--flake8"), but this time observe some tests failing due to flake8.
Details (click to expand):
```
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <_pytest.config.PytestPluginManager object at 0x1052391e0>
hook_name = 'pytest_runtest_call'
methods = [>, ...]
kwargs = {'item': }, firstresult = False
def _hookexec(self, hook_name, methods, kwargs, firstresult):
# called from all hookcaller instances.
# enable_tracing will set its own wrapping function at self._inner_hookexec
> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
../../../mambaforge/envs/daskimg/lib/python3.10/site-packages/pluggy/_manager.py:80:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hook_name = 'pytest_runtest_call'
hook_impls = [>, ...]
caller_kwargs = {'item': }, firstresult = False
def _multicall(hook_name, hook_impls, caller_kwargs, firstresult):
"""Execute a call into multiple python functions/methods and return the
result(s).
``caller_kwargs`` comes from _HookCaller.__call__().
"""
__tracebackhide__ = True
results = []
excinfo = None
try: # run impl and wrapper setup functions in a loop
teardowns = []
try:
for hook_impl in reversed(hook_impls):
try:
args = [caller_kwargs[argname] for argname in hook_impl.argnames]
except KeyError:
for argname in hook_impl.argnames:
if argname not in caller_kwargs:
raise HookCallError(
f"hook call must provide argument {argname!r}"
)
if hook_impl.hookwrapper:
try:
gen = hook_impl.function(*args)
next(gen) # first yield
teardowns.append(gen)
except StopIteration:
_raise_wrapfail(gen, "did not yield")
else:
res = hook_impl.function(*args)
if res is not None:
results.append(res)
if firstresult: # halt further impl calls
break
except BaseException:
excinfo = sys.exc_info()
finally:
if firstresult: # first result hooks return a single value
outcome = _Result(results[0] if results else None, excinfo)
else:
outcome = _Result(results, excinfo)
# run all wrapper post-yield blocks
for gen in reversed(teardowns):
try:
gen.send(outcome)
_raise_wrapfail(gen, "has second yield")
except StopIteration:
pass
> return outcome.get_result()
../../../mambaforge/envs/daskimg/lib/python3.10/site-packages/pluggy/_callers.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
def get_result(self):
"""Get the result(s) for this hook call.
If the hook was marked as a ``firstresult`` only a single value
will be returned otherwise a list of results.
"""
__tracebackhide__ = True
if self._excinfo is None:
return self._result
else:
ex = self._excinfo
> raise ex[1].with_traceback(ex[2])
../../../mambaforge/envs/daskimg/lib/python3.10/site-packages/pluggy/_result.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hook_name = 'pytest_runtest_call'
hook_impls = [>, ...]
caller_kwargs = {'item': }, firstresult = False
def _multicall(hook_name, hook_impls, caller_kwargs, firstresult):
"""Execute a call into multiple python functions/methods and return the
result(s).
``caller_kwargs`` comes from _HookCaller.__call__().
"""
__tracebackhide__ = True
results = []
excinfo = None
try: # run impl and wrapper setup functions in a loop
teardowns = []
try:
for hook_impl in reversed(hook_impls):
try:
args = [caller_kwargs[argname] for argname in hook_impl.argnames]
except KeyError:
for argname in hook_impl.argnames:
if argname not in caller_kwargs:
raise HookCallError(
f"hook call must provide argument {argname!r}"
)
if hook_impl.hookwrapper:
try:
gen = hook_impl.function(*args)
next(gen) # first yield
teardowns.append(gen)
except StopIteration:
_raise_wrapfail(gen, "did not yield")
else:
> res = hook_impl.function(*args)
../../../mambaforge/envs/daskimg/lib/python3.10/site-packages/pluggy/_callers.py:39:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
item =
def pytest_runtest_call(item: Item) -> None:
_update_current_test_var(item, "call")
try:
del sys.last_type
del sys.last_value
del sys.last_traceback
except AttributeError:
pass
try:
item.runtest()
except Exception as e:
# Store trace info to allow postmortem debugging
sys.last_type = type(e)
sys.last_value = e
assert e.__traceback__ is not None
# Skip *this* frame
sys.last_traceback = e.__traceback__.tb_next
> raise e
../../../mambaforge/envs/daskimg/lib/python3.10/site-packages/_pytest/runner.py:175:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
item =
def pytest_runtest_call(item: Item) -> None:
_update_current_test_var(item, "call")
try:
del sys.last_type
del sys.last_value
del sys.last_traceback
except AttributeError:
pass
try:
> item.runtest()
../../../mambaforge/envs/daskimg/lib/python3.10/site-packages/_pytest/runner.py:167:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
def runtest(self):
with BytesIO() as bo, TextIOWrapper(bo, encoding='utf-8') as to, \
BytesIO() as be, TextIOWrapper(be, encoding='utf-8') as te, \
redirect_stdout(to), redirect_stderr(te):
> found_errors = check_file(
self.fspath,
self.flake8ignore,
self.maxlength,
self.maxdoclength,
self.maxcomplexity,
self.showsource,
self.statistics
)
../../../mambaforge/envs/daskimg/lib/python3.10/site-packages/pytest_flake8.py:136:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
path = local('/Users/genevieb/Documents/GitHub/dask-image/tests/test_dask_image/test_ndmorph/test_ndmorph.py')
flake8ignore = [], maxlength = '', maxdoclenght = '', maxcomplexity = ''
showsource = [], statistics = []
def check_file(path, flake8ignore, maxlength, maxdoclenght, maxcomplexity,
showsource, statistics):
"""Run flake8 over a single file, and return the number of failures."""
args = []
if maxlength:
args += ['--max-line-length', maxlength]
if maxdoclenght:
args += ['--max-doc-length', maxdoclenght]
if maxcomplexity:
args += ['--max-complexity', maxcomplexity]
if showsource:
args += ['--show-source']
if statistics:
args += ['--statistics']
app = application.Application()
> prelim_opts, remaining_args = app.parse_preliminary_options(args)
E AttributeError: 'Application' object has no attribute 'parse_preliminary_options'
../../../mambaforge/envs/daskimg/lib/python3.10/site-packages/pytest_flake8.py:216: AttributeError
FAILED dask_image/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/_version.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/dispatch/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/dispatch/_dispatch_ndfilters.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/dispatch/_dispatch_ndinterp.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/dispatch/_dispatch_ndmorph.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/dispatch/_dispatcher.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/dispatch/_utils.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/imread/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/_conv.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/_diff.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/_edge.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/_gaussian.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/_generic.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/_order.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/_smooth.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/_threshold.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfilters/_utils.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfourier/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndfourier/_utils.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndinterp/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndmeasure/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndmeasure/_utils/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndmeasure/_utils/_find_objects.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndmeasure/_utils/_label.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndmorph/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndmorph/_ops.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED dask_image/ndmorph/_utils.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED docs/conf.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED docs/release/generate_release_notes.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_imread/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_imread/test_core.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_imread/test_cupy_imread.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test__conv.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test__diff.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test__edge.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test__gaussian.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test__generic.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test__order.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test__smooth.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test__threshold.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test__utils.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test_cupy_ndfilters.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfilters/test_cupy_threshold.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfourier/test__utils.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndfourier/test_core.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndinterp/test_affine_transformation.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndinterp/test_spline_filter.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndmeasure/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndmeasure/test__utils.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndmeasure/test_core.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndmeasure/test_find_objects.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndmorph/__init__.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndmorph/test__utils.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndmorph/test_cupy_ndmorph.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
FAILED tests/test_dask_image/test_ndmorph/test_ndmorph.py::flake-8::FLAKE8 - AttributeError: 'Application' object has no attribute 'parse_preliminary_op...
```
**Anything else we need to know?**:
I've noticed the same problem in https://github.com/dask/dask-image/pull/306
I might have done something silly by adding a separate `pytest.ini` file in https://github.com/dask/dask-image/pull/151, which might override or conflict with the pytest section in `setup.cfg`. So this might have masked the problem for a while, but since I see the same problems with the new pyrpoject.toml configuration (https://github.com/dask/dask-image/pull/306) that can't explain everything.
**Environment**:
- Dask version: 2023.03.2
- dask-image version: main branch version (2022.9.0+30.gb08d494)
- Python version: 3.10
- Operating System: Mac Monterey
- Install method (conda, pip, source): pip installed into a conda virtual environment
conda list (click to expand):
```
# Name Version Build Channel
appnope 0.1.3 pyhd8ed1ab_0 conda-forge
asttokens 2.2.1 pyhd8ed1ab_0 conda-forge
attrs 22.2.0 pypi_0 pypi
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
bzip2 1.0.8 h3422bc3_4 conda-forge
ca-certificates 2022.12.7 h4653dfc_0 conda-forge
click 8.1.3 pypi_0 pypi
cloudpickle 2.2.1 pypi_0 pypi
dask 2023.3.2 pypi_0 pypi
dask-image 2022.9.0+30.gb08d494 dev_0
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.1.1 pypi_0 pypi
executing 1.2.0 pyhd8ed1ab_0 conda-forge
flake8 6.0.0 pypi_0 pypi
fsspec 2023.3.0 pypi_0 pypi
imageio 2.27.0 pypi_0 pypi
importlib-metadata 6.1.0 pypi_0 pypi
iniconfig 2.0.0 pypi_0 pypi
ipython 8.11.0 pyhd1c38e8_0 conda-forge
jedi 0.18.2 pyhd8ed1ab_0 conda-forge
libffi 3.4.2 h3422bc3_5 conda-forge
libsqlite 3.40.0 h76d750c_0 conda-forge
libzlib 1.2.13 h03a7124_4 conda-forge
locket 1.0.0 pypi_0 pypi
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mccabe 0.7.0 pypi_0 pypi
ncurses 6.3 h07bb92c_1 conda-forge
numpy 1.24.2 pypi_0 pypi
openssl 3.1.0 h03a7124_0 conda-forge
packaging 23.0 pypi_0 pypi
pandas 1.5.3 pypi_0 pypi
parso 0.8.3 pyhd8ed1ab_0 conda-forge
partd 1.3.0 pypi_0 pypi
pexpect 4.8.0 pyh1a96a4e_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 9.4.0 pypi_0 pypi
pims 0.6.1 pypi_0 pypi
pip 23.0.1 pyhd8ed1ab_0 conda-forge
pluggy 1.0.0 pypi_0 pypi
prompt-toolkit 3.0.38 pyha770c72_0 conda-forge
prompt_toolkit 3.0.38 hd8ed1ab_0 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pycodestyle 2.10.0 pypi_0 pypi
pyflakes 3.0.1 pypi_0 pypi
pygments 2.14.0 pyhd8ed1ab_0 conda-forge
pytest 7.2.2 pypi_0 pypi
pytest-flake8 1.1.1 pypi_0 pypi
pytest-timeout 2.1.0 pypi_0 pypi
python 3.10.10 h3ba56d0_0_cpython conda-forge
python-dateutil 2.8.2 pypi_0 pypi
pytz 2023.3 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
readline 8.2 h92ec313_1 conda-forge
scipy 1.10.1 pypi_0 pypi
setuptools 67.6.1 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
slicerator 1.1.0 pypi_0 pypi
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
tifffile 2023.3.21 pypi_0 pypi
tk 8.6.12 he1e0b03_0 conda-forge
tomli 2.0.1 pypi_0 pypi
toolz 0.12.0 pypi_0 pypi
traitlets 5.9.0 pyhd8ed1ab_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge
wheel 0.40.0 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h57fd34a_0 conda-forge
zipp 3.15.0 pypi_0 pypi
```
Contributor guide
Research direction
Start by running pytest and pytest --flake8 in a development installation with the test requirements, then compare their handling of the [tool:pytest] addopts setting. Inspect the pytest_flake8 check_file entry point and the reported Application.parse_preliminary_options failure. Done means pytest applies the configured flake8 check and pytest --flake8 completes without the reported hook errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100