Interpreter is not found when using Anaconda
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 16.2k
- Forks
- 1k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 66
Description
🐛 Bug Reports
When reporting a bug, please provide the following information. If this is not a bug report you can just discard this template.
🌍 Environment
- Your operating system and version: Windows 10
- Your python version: 3.8.5
- How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: Anaconda default Python installation
- Your Rust version (
rustc --version): 1.53.0-nightly - Your PyO3 version: 0.13.2
- Have you tried using latest PyO3 main (replace
version = "0.x.y"withgit = "https://github.com/PyO3/pyo3")?: Yes
💥 Reproducing
Please provide a minimal working example. This means both the Rust code and the Python.
The following code compiles, but crashes when using Anaconda distribution (using the default installation).
use pyo3::prelude::*;
fn main() -> PyResult<()>{
let gil = Python::acquire_gil();
let py = gil.python();
let result = py.run("print('it works')", None, None);
if let Err(ref err) = result {
println!("{:?}", err);
}
Ok(())
}
The output I got:
Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = (not set)
program name = 'python'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = 'C:\\bug\\target\\debug\\bug.exe'
sys.base_prefix = 'C:\\Users\\balog\\anaconda3'
sys.base_exec_prefix = 'C:\\Users\\balog\\anaconda3'
sys.executable = 'C:\\bug\\target\\debug\\bug.exe'
sys.prefix = 'C:\\Users\\balog\\anaconda3'
sys.exec_prefix = 'C:\\Users\\balog\\anaconda3'
sys.path = [
'C:\\Users\\balog\\anaconda3\\python38.zip',
'.\\DLLs',
'.\\lib',
'C:\\bug\\target\\debug',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00000e6c (most recent call first):
<no Python frame>
error: process didn't exit successfully: `target\debug\bug.exe` (exit code: 1)
The sys executables are pointing to wrong locations, so it seems it couldn't find the Python interpreter. I haven't checked how PyO3 tries to find the interpreter internally, but as far as I know Anaconda doesn't set PYTHONPATH nor PYTHONHOME env variables and that might be the cause that breaks it. Anaconda sets up the CONDA_PREFIX, which can be - at least in my case - set as PYTHONHOME, so it's my temporal workaround:
use pyo3::ffi;
use pyo3::prelude::*;
use widestring::WideCString;
fn main() -> PyResult<()> {
if let Some(PYTHONHOME) = std::env::var_os("CONDA_PREFIX") {
unsafe {
ffi::Py_SetPythonHome(
WideCString::from_str(PYTHONHOME.to_str().unwrap())
.unwrap()
.as_ptr(),
);
}
}
let gil = Python::acquire_gil();
let py = gil.python();
let result = py.run("print('it works')", None, None);
if let Err(ref err) = result {
println!("{:?}", err);
}
Ok(())
}
However I'm not sure it works out-of-the-box.
The output of conda info -a:
active environment : base
active env location : C:\Users\balog\anaconda3
shell level : 1
user config file : C:\Users\balog\.condarc
populated config files :
conda version : 4.9.2
conda-build version : 3.20.5
python version : 3.8.5.final.0
virtual packages : __cuda=9.1=0
__win=0=0
__archspec=1=x86_64
base environment : C:\Users\balog\anaconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : C:\Users\balog\anaconda3\pkgs
C:\Users\balog\.conda\pkgs
C:\Users\balog\AppData\Local\conda\conda\pkgs
envs directories : C:\Users\balog\anaconda3\envs
C:\Users\balog\.conda\envs
C:\Users\balog\AppData\Local\conda\conda\envs
platform : win-64
user-agent : conda/4.9.2 requests/2.24.0 CPython/3.8.5 Windows/10 Windows/10.0.19041
administrator : False
netrc file : None
offline mode : False
# conda environments:
#
base * C:\Users\balog\anaconda3
sys.version: 3.8.5 (default, Sep 3 2020, 21:29:08) [...
sys.prefix: C:\Users\balog\anaconda3
sys.executable: C:\Users\balog\anaconda3\python.exe
conda location: C:\Users\balog\anaconda3\lib\site-packages\conda
conda-build: C:\Users\balog\anaconda3\Scripts\conda-build.exe
conda-convert: C:\Users\balog\anaconda3\Scripts\conda-convert.exe
conda-debug: C:\Users\balog\anaconda3\Scripts\conda-debug.exe
conda-develop: C:\Users\balog\anaconda3\Scripts\conda-develop.exe
conda-env: C:\Users\balog\anaconda3\Scripts\conda-env.exe
conda-index: C:\Users\balog\anaconda3\Scripts\conda-index.exe
conda-inspect: C:\Users\balog\anaconda3\Scripts\conda-inspect.exe
conda-metapackage: C:\Users\balog\anaconda3\Scripts\conda-metapackage.exe
conda-render: C:\Users\balog\anaconda3\Scripts\conda-render.exe
conda-server: C:\Users\balog\anaconda3\Scripts\conda-server.exe
conda-skeleton: C:\Users\balog\anaconda3\Scripts\conda-skeleton.exe
conda-verify: C:\Users\balog\anaconda3\Scripts\conda-verify.exe
user site dirs:
CIO_TEST: <not set>
CONDA_DEFAULT_ENV: base
CONDA_EXE: C:\Users\balog\anaconda3\condabin\..\Scripts\conda.exe
CONDA_EXES: "C:\Users\balog\anaconda3\condabin\..\Scripts\conda.exe"
CONDA_PREFIX: C:\Users\balog\anaconda3
CONDA_PROMPT_MODIFIER: (base)
CONDA_PYTHON_EXE: C:\Users\balog\anaconda3\python.exe
CONDA_ROOT: C:\Users\balog\anaconda3
CONDA_SHLVL: 1
CURL_CA_BUNDLE: <not set>
HOMEPATH: \Users\balog
PATH: C:\Users\balog\anaconda3;C:\Users\balog\anaconda3\Library\mingw-w64\bin;C:\Users\balog\anaconda3\Library\usr\bin;C:\Users\balog\anaconda3\Library\bin;C:\Users\balog\anaconda3\Scripts;C:\Users\balog\anaconda3\bin;C:\Users\balog\anaconda3;C:\Users\balog\anaconda3\Library\mingw-w64\bin;C:\Users\balog\anaconda3\Library\usr\bin;C:\Users\balog\anaconda3\Library\bin;C:\Users\balog\anaconda3\Scripts;C:\Users\balog\anaconda3\bin;C:\Users\balog\anaconda3\condabin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\pysprint-cli\bin;C:\Users\balog\.cargo\bin;C:\Users\balog\AppData\Local\Microsoft\WindowsApps;C:\Users\balog\anaconda3\python.exe;.
PSMODULEPATH: C:\Program Files\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules
REQUESTS_CA_BUNDLE: <not set>
SSL_CERT_FILE: <not set>
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the minimal Rust example on Windows with the reported Anaconda installation, then inspect PyO3's Python interpreter initialization and path configuration. Done means the example runs with Anaconda without requiring the manual CONDA_PREFIX/PYTHONHOME workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- anaconda, python, rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100