python / python/cpython

Importing numpy fails when CPython is compiled to a DLL

未關閉
#92,282 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

OS-windows type-bug
主要語言
Python
星號
77.2k
分支
36k
PR 合併指標
PR 指標待擷取

描述

I am trying to use numpy 1.22.3 with embedded python 3.9.12 or 3.10.2 compiled to a DLL on Windows 10.0.19044.

I've added numpy to embedded python by first installing it to the system python of the same version, then copying the new folders from Lib/site-packages to python3x.zip/site-packages. I have also installed numpy by first installing pip with get-pip from PyPa, but get the same results.

Importing numpy fails in three ways:

When compiling to a UWP DLL or standard DLL without unzipping python3x.zip, I get the following error:

ImportError('
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed.
We have compiled some common reasons and troubleshooting tips at: https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.10 from "C:\\Users\\jtemplin\\git\\PythonInterpreterDLL\\x64\\Debug\\PythonInterpreterTests\\AppX\\PythonInterpreterTests.exe"
* The NumPy version is: "1.22.3"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named \'numpy.core._multiarray_umath\'
')

This happens in debug and release builds. Other recommendations on the linked troubleshooting page either didn't apply or made no difference.

However, if I unzip the python3x.zip I get different errors:

When compiling to a UWP DLL and and importing numpy in python using PyObject_CallObject I get the following error:

Traceback (most recent call last):
File "C:\path-to-build\tests.py", line 37, in import_numpy_exception
import numpy
File "C:\path-to-build\python310.zip\site-packages\numpy\__init__.py", line 128, in <module>
from numpy.__config__ import show as show_config
File "C:\path-to-build\python310.zip\site-packages\numpy\__config__.py", line 13, in <module>
os.add_dll_directory(extra_dll_dir)
File "os.py", line 1117, in add_dll_directory
OSError: [WinError 87] The parameter is incorrect: 'C:\\path-to-build\\python310.zip\\site-packages\\numpy\\.libs'

The path that os.add_dll_directory is trying to add exists, and the function works if I call it manually in a python shell.

When compiling to a standard DLL I can call os.add_dll_directory using PyRun_SimpleString, but still can't import numpy. I've tried PyImport_Import("numpy"), PyImport_ImportModule("numpy"), PyImport_AddModule("numpy") and they all return nullptr. I've also tried importing numpy in python using PyObject_CallObject and it fails with the following error:
Assertion failed: !PyErr_Occurred(), file D:\a\1\s\Objects\typeobject.c, line 3264

All of the above functions are able to import django successfully, so I don't believe it is a simple path issue. They also import numpy successfully when running in a console app.

So why does importing numpy fail in a DLL?

Here are some code samples that work with django, but not numpy:

bool PythonInterpreter::importModule(const char* moduleName) {
    // moduleName = "numpy" or "django"
    auto pyModuleName = PyUnicode_FromString(moduleName);
    auto module = PyImport_Import(pyModuleName);
    auto result1 = !(nullptr == module);
    
    auto module2 = PyImport_ImportModule(moduleName);
    auto result2 = !(nullptr == module2);
    
    auto module3 = PyImport_AddModule(moduleName);
    auto result3 = !(nullptr == module3);
    
    return result1 || result2 || result3;
}

bool PythonInterpreter::importSimpleString(const char* importString) {
    // importString = "import numpy" or "import django"
    auto result = 0 == PyRun_SimpleString(importString);
    
    return result;
}

bool PythonInterpreter::importWithException(const char* functionName) {
    // functionName = "import_numpy" or "import_django"
    auto importNumpyEx = PyObject_GetAttrString(this->testModule, functionName);
    auto result = _PyUnicode_AsString(PyObject_CallObject(importNumpyEx, NULL));
    
    return 0 == strcmp(result, "PASS");
}
def import_django():
    try:
        import django
        return "PASS"
    except OSError as e:
        return full_stack() # lifted from stackexchange

def import_numpy():
    try:
        import numpy
        return "PASS"
    except OSError as e:
        return full_stack() # lifted from stackexchange

def full_stack():
    import traceback, sys
    exc = sys.exc_info()[0]
    stack = traceback.extract_stack()[:-1]  # last one would be full_stack()
    if exc is not None:  # i.e. an exception is present
        del stack[-1]       # remove call of full_stack, the printed exception
                            # will contain the caught exception caller instead
    trc = 'Traceback (most recent call last):\n'
    stackstr = trc + ''.join(traceback.format_list(stack))
    if exc is not None:
         stackstr += '  ' + traceback.format_exc().lstrip(trc)
    return

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

未指定 CPython 檔案或測試。首先,在 DLL 和主控台組態中使用所示的進入點 PyImport_Import、PyRun_SimpleString 和 PyObject_CallObject 重現此失敗,並將行為與 django 進行比較。完成的標準是確認責任在於 CPython 還是嵌入設定,並記錄可重現的診斷結果。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
cpp, numpy, python
領域
operating-systems
Issue 類型
缺陷
難度
5/5
預估耗時
一週以上
活躍度
停滯
描述清晰度
需要釐清
新手友好度
25/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。