pytest-dev / pytest-dev/pytest

Pytest using matplotlib finding numpy in local Win10 env when run in WSL environment.

Open
#6,678 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

platform: windows topic: collection type: bug
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Bug report

https://github.com/numpy/numpy/issues/15520

I'm running pytest on a project in WSL Ubuntu, and I'm getting an error with numpy. Matplotlib is being found in the WSL python install, but that matplotlib finds numpy in the Windows env Pycharm has in my project folder:

PycharmProjects/
    project_folder/
        project_source/
        tests_folder/
        windows_project_env/
        README.md
        app_main.py

I'll note that the project works fine, but this is possibly because it is not using matplotlib/numpy in a way that that causes this issue.

This is just a regular python3.8 install inside Ubuntu. I assumed it was an environment issue but every imported module when run via REPL, script, and tests in pytest has their .__file__ returning a path in the regular site packages, including numpy and matplotlib, when called directly import numpy; numpy.__file__, but it errors when matplotlib calls numpy (or when pytest scans through matpotlib and follows it's links to numpy while it's doing test discovery) because it's finding numpy in a Windows Env (according to the stack trace) and the extensions are different etc.

When run in the project folder via `python3.8 -m pytest tests_folder/ it works fine. So my assumption is that this is something to do with how pytest looks for tests? The application runs fine, although it's probably not calling the numpy extensions in question.

Stack trace:

toonarmycaptain@DESKTOP-05F10IK:/mnt/c/Users/david/OneDrive/Programming/PycharmProjects/dionysus$ python3.8 -m pytest
================================================= test session starts ==================================================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /mnt/c/Users/david/OneDrive/Programming/PycharmProjects/dionysus, inifile: pytest.ini
plugins: cov-2.8.1, mypy-0.4.2
collected 0 items / 1 error

======================================================== ERRORS ========================================================
____________________________________________ ERROR collecting test session _____________________________________________
/home/toonarmycaptain/.local/lib/python3.8/site-packages/_pytest/config/__init__.py:459: in _importconftest
    return self._conftestpath2mod[key]
E   KeyError: PosixPath('/mnt/c/Users/david/OneDrive/Programming/PycharmProjects/dionysus/dionysus_env/Lib/site-packages/numpy/conftest.py')

During handling of the above exception, another exception occurred:
dionysus_env/Lib/site-packages/numpy/core/__init__.py:17: in <module>
    from . import multiarray
dionysus_env/Lib/site-packages/numpy/core/multiarray.py:14: in <module>
    from . import overrides
dionysus_env/Lib/site-packages/numpy/core/overrides.py:7: in <module>
    from numpy.core._multiarray_umath import (
E   ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:
/home/toonarmycaptain/.local/lib/python3.8/site-packages/_pytest/config/__init__.py:465: in _importconftest
    mod = conftestpath.pyimport()
/home/toonarmycaptain/.local/lib/python3.8/site-packages/py/_path/local.py:701: in pyimport
    __import__(modname)
dionysus_env/Lib/site-packages/numpy/__init__.py:142: in <module>
    from . import core
dionysus_env/Lib/site-packages/numpy/core/__init__.py:47: in <module>
    raise ImportError(msg)
E   ImportError:
E
E   IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
E
E   Importing the numpy c-extensions failed.
E   - Try uninstalling and reinstalling numpy.
E   - If you have already done that, then:
E     1. Check that you expected to use Python3.8 from "/usr/bin/python3.8",
E        and that you have no directories in your PATH or PYTHONPATH that can
E        interfere with the Python and numpy version "1.17.3" you're trying to use.
E     2. If (1) looks fine, you can open a new issue at
E        https://github.com/numpy/numpy/issues.  Please include details on:
E        - how you installed Python
E        - how you installed numpy
E        - your operating system
E        - whether or not you have multiple versions of Python installed
E        - if you built from source, your compiler versions and ideally a build log
E
E   - If you're working with a numpy git repository, try `git clean -xdf`
E     (removes all files not under version control) and rebuild numpy.
E
E   Note: this error has many possible causes, so please don't comment on
E   an existing issue about this - open a new one instead.
E
E   Original error was: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:
/home/toonarmycaptain/.local/lib/python3.8/site-packages/py/_path/common.py:383: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
/home/toonarmycaptain/.local/lib/python3.8/site-packages/py/_path/common.py:435: in gen
    for p in self.gen(subdir):
/home/toonarmycaptain/.local/lib/python3.8/site-packages/py/_path/common.py:435: in gen
    for p in self.gen(subdir):
/home/toonarmycaptain/.local/lib/python3.8/site-packages/py/_path/common.py:435: in gen
    for p in self.gen(subdir):
/home/toonarmycaptain/.local/lib/python3.8/site-packages/py/_path/common.py:424: in gen
    dirs = self.optsort([p for p in entries
/home/toonarmycaptain/.local/lib/python3.8/site-packages/py/_path/common.py:425: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
/home/toonarmycaptain/.local/lib/python3.8/site-packages/_pytest/main.py:626: in _recurse
    ihook = self.gethookproxy(dirpath)
/home/toonarmycaptain/.local/lib/python3.8/site-packages/_pytest/main.py:445: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
/home/toonarmycaptain/.local/lib/python3.8/site-packages/_pytest/config/__init__.py:437: in _getconftestmodules
    mod = self._importconftest(conftestpath)
/home/toonarmycaptain/.local/lib/python3.8/site-packages/_pytest/config/__init__.py:473: in _importconftest
    raise ConftestImportFailure(conftestpath, sys.exc_info())
E   _pytest.config.ConftestImportFailure: (local('/mnt/c/Users/david/OneDrive/Programming/PycharmProjects/dionysus/dionysus_env/Lib/site-packages/numpy/conftest.py'), (<class 'ImportError'>, ImportError('\n\nIMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!\n\nImporting the numpy c-extensions failed.\n- Try uninstalling and reinstalling numpy.\n- If you have already done that, then:\n  1. Check that you expected to use Python3.8 from "/usr/bin/python3.8",\n     and that you have no directories in your PATH or PYTHONPATH that can\n     interfere with the Python and numpy version "1.17.3" you\'re trying to use.\n  2. If (1) looks fine, you can open a new issue at\n     https://github.com/numpy/numpy/issues.  Please include details on:\n     - how you installed Python\n     - how you installed numpy\n     - your operating system\n     - whether or not you have multiple versions of Python installed\n     - if you built from source, your compiler versions and ideally a build log\n\n- If you\'re working with a numpy git repository, try `git clean -xdf`\n  (removes all files not under version control) and rebuild numpy.\n\nNote: this error has many possible causes, so please don\'t comment on\nan existing issue about this - open a new one instead.\n\nOriginal error was: No module named \'numpy.core._multiarray_umath\'\n'), <traceback object at 0x7ff7c2a76900>))
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=================================================== 1 error in 3.10s ===================================================

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pytest invocation python3.8 -m pytest and the collection path in the traceback, especially _pytest/config/__init__.py and _pytest/main.py. Reproduce the WSL project layout with the Windows environment present, then trace why collection imports dionysus_env/Lib/site-packages/numpy/conftest.py; done means pytest no longer follows that foreign environment during collection and the regression is covered by a test.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, python, ubuntu
Domain
operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.