PyO3 / PyO3/pyo3

Can not link to virtual environment on OS X

Open
#1,741 19 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
16.2k
Forks
1k
Avg merge
2d 6h
Merged PRs (30d)
66

Description

🐛 Bug Reports

I can not link to the correct virtual environment in OS X. Instead, the system Python is always used.

🌍 Environment
  • Your operating system and version: OS X Big Sur 11.5
  • Your python version: 3.8.11
  • How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: Python was installed with homebrew via brew install python@3.8 A virtual environment was created with python3 -m venv redvox and is located at /Users/anthony/venvs/redvox
  • Your Rust version (rustc --version): rustc 1.56.0-nightly (9c25eb7aa 2021-07-25)
  • Your PyO3 version: 14.1
  • Have you tried using latest PyO3 main (replace version = "0.x.y" with git = "https://github.com/PyO3/pyo3")?:

No

💥 Reproducing
  1. Activate the virtual environment source /Users/anthony/venvs/redvox/bin/activate
  2. Ensure matplotlib is installed pip install matplotlib
  3. Run the rust code cargo run
use pyo3::prelude::*;
use std::io::Result;

fn main() -> Result<()> {
    let src = r#"
import sys
print(sys.executable, sys.path)
import matplotlib
            "#;

    Python::with_gil(|py| {
        py.run(src, None, None).unwrap();
    });

    Ok(())
}

The following shows how I activate the virtual environment and attempt to call the rust code. The error message I receive is also included.

anthony@REDVOX-00-MACBOOK-PRO rust-sandbox % source ~/venvs/redvox/bin/activate
(redvox) anthony@REDVOX-00-MACBOOK-PRO rust-sandbox % python3
Python 3.8.11 (default, Jun 29 2021, 03:08:07) 
[Clang 12.0.5 (clang-1205.0.22.9)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
>>> exit()
(redvox) anthony@REDVOX-00-MACBOOK-PRO rust-sandbox % cargo run 
   Compiling pyo3 v0.14.1
   Compiling rust-sandbox v0.1.0 (/Users/anthony/Development/rust-sandbox)
    Finished dev [unoptimized + debuginfo] target(s) in 4.63s
     Running `target/debug/rust-sandbox`
/Users/anthony/Development/rust-sandbox/target/debug/rust-sandbox ['/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python38.zip', '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8', '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload', '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages']
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'ModuleNotFoundError'>, value: ModuleNotFoundError("No module named 'matplotlib'"), traceback: Some(<traceback object at 0x100ddbbc0>) }', src/main.rs:12:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I've tried setting PYTHONHOME and PYTHONPATH. Neither of these have worked for me. The system Python is always used no matter what I try. How can I get pyo3 to work with my virtual environment on OS X?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproducer in src/main.rs and run it with the stated macOS, Python, Rust, and PyO3 versions. Trace how Python::with_gil initializes the interpreter and selects its environment. Done means the embedded interpreter uses the activated virtual environment and the matplotlib import succeeds; verify with cargo run.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.