indygreg / indygreg/PyOxidizer

Unable to Install Numpy on M1

Open
#657 5 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
6.2k
Forks
256
PR merge metrics
No merged PRs in 30d

Description

Hello, I am currently using pyoxidizer version 0.23.0 on a mac m1 to bundle numpy; however, I consistently run into an error when importing numpy into my environment. Here is my error message:

installing files to /Users/nate_vandenhoek/Projects/PyOx_Test/testing/./build/aarch64-apple-darwin/debug/install
Python 3.9.15 (main, Nov  6 2022, 11:31:59)
[Clang 14.0.3 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
  File "/Users/nate_vandenhoek/Projects/PyOx_Test/testing/build/aarch64-apple-darwin/debug/install/lib/numpy/core/__init__.py", line 23, in <module>
    from . import multiarray
  File "/Users/nate_vandenhoek/Projects/PyOx_Test/testing/build/aarch64-apple-darwin/debug/install/lib/numpy/core/multiarray.py", line 10, in <module>
    from . import overrides
  File "/Users/nate_vandenhoek/Projects/PyOx_Test/testing/build/aarch64-apple-darwin/debug/install/lib/numpy/core/overrides.py", line 6, in <module>
    from numpy.core._multiarray_umath import (
ImportError: dlopen(/Users/nate_vandenhoek/Projects/PyOx_Test/testing/build/aarch64-apple-darwin/debug/install/lib/numpy/core/_multiarray_umath.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace (_PyCapsule_GetContext)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/nate_vandenhoek/Projects/PyOx_Test/testing/build/aarch64-apple-darwin/debug/install/lib/numpy/__init__.py", line 140, in <module>
    from . import core
  File "/Users/nate_vandenhoek/Projects/PyOx_Test/testing/build/aarch64-apple-darwin/debug/install/lib/numpy/core/__init__.py", line 49, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "/Users/nate_vandenhoek/Projects/PyOx_Test/testing/build/aarch64-apple-darwin/debug/install/numpy"
  * The NumPy version is: "1.23.0"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/Users/nate_vandenhoek/Projects/PyOx_Test/test/testing/aarch64-apple-darwin/debug/install/lib/numpy/core/_multiarray_umath.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace (_PyCapsule_GetContext)

I have followed the instructions found here , but still seem to be running into this issue. I have scoured the internet for examples, but to no avail. Can you help me understand what I am missing? Any insight you could provide is appreciated. Thanks!

Here is my pyoxidizer.bzl file:

def make_exe():
    dist = default_python_distribution(python_version="3.9")
    

    policy = dist.make_python_packaging_policy()
    policy.set_resource_handling_mode("files")
    policy.resources_location_fallback = "filesystem-relative:lib"

    python_config = dist.make_python_interpreter_config()
    python_config.module_search_paths = ["$ORIGIN/lib"]

    exe = dist.to_python_executable(
        name="numpy",
        packaging_policy=policy,
        config=python_config,
    )

    # exe.add_python_resources(exe.pip_install(["numpy"]))

    for resource in exe.pip_download(["numpy==1.23.0"]):
        resource.add_location = "filesystem-relative:lib"
        exe.add_python_resource(resource)
    return exe

def make_embedded_resources(exe):
    return exe.to_embedded_resources()

def make_install(exe):
    # Create an object that represents our installed application file layout.
    files = FileManifest()

    # Add the generated executable to our install layout in the root directory.
    files.add_python_resource(".", exe)

    return files

def make_msi(exe):
    # See the full docs for more. But this will convert your Python executable
    # into a `WiXMSIBuilder` Starlark type, which will be converted to a Windows
    # .msi installer when it is built.
    return exe.to_wix_msi_builder(
        # Simple identifier of your app.
        "myapp",
        # The name of your application.
        "My Application",
        # The version of your application.
        "1.0",
        # The author/manufacturer of your application.
        "Alice Jones"
    )


# Dynamically enable automatic code signing.
def register_code_signers():
    # You will need to run with `pyoxidizer build --var ENABLE_CODE_SIGNING 1` for
    # this if block to be evaluated.
    if not VARS.get("ENABLE_CODE_SIGNING"):
        return

# Call our function to set up automatic code signers.
register_code_signers()

# Tell PyOxidizer about the build targets defined above.
register_target("exe", make_exe)
register_target("resources", make_embedded_resources, depends=["exe"], default_build_script=True)
register_target("install", make_install, depends=["exe"], default=True)
register_target("msi_installer", make_msi, depends=["exe"])

# Resolve whatever targets the invoker of this configuration file is requesting
# be resolved.
resolve_targets()

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pyoxidizer.bzl configuration and reproduce the reported import numpy failure on the aarch64-apple-darwin build described in the issue. Inspect the filesystem resource handling and the _multiarray_umath loading error; done means the bundled application imports NumPy without the missing _PyCapsule_GetContext symbol error.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, numpy, python
Domain
build-system, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.