huggingface / huggingface/candle

UV is incompatible with pyo3 for the reinforcement-learning example

Open
#2,844 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

If you create a venv using uv and pip install gymnasium, the example command (`cargo run --example reinforcement-learning --features=pyo3 --package candle-examples -- pg`) will not find any of your installed libraries.

There's a discussion thread about workarounds [here](https://github.com/PyO3/pyo3/issues/4813#issuecomment-2617037553), but the solution of
```
$ find ~/.local/share/uv/python -name "libpython3.xx.dylib"
> /path/to/pythondir/lib/libpython3.xx.dylib
DYLD_FALLBACK_LIBRARY_PATH=/path/to/pythondir/lib/ CARGO_BUILD_RUSTFLAGS="-L /path/to/pythondir/lib/" cargo run --example reinforcement-learning --features=pyo3 --package candle-examples -- pg
```
Gives an error
```
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00000001f8918840 (most recent call first):

```
Which is failing because it is searching for python libs in `\install` which doesn't exist.

## My solution (Mac OS)

This solution is pretty brittle, but it works. It renames the install names of the python libs to point at itself instead of the `/install` directory.

```bash
$ uv venv
$ source .venv/bin/activate
$ uv pip install gymnasium
$ which python3.xx
> /path/to/.venv/bin/python3.xx
$ find ~/.local/share/uv/python -name "libpython3.xx.dylib"
> /path/to/pythondir/lib/libpython3.xx.dylib
$ cp /path/to/pythondir/lib/libpython3.xx.dylib ~/libpython3.xx.dylib.backup
$ install_name_tool -id "/path/to/pythondir/lib/libpython3.xx.dylib" "/path/to/pythondir/lib/libpython3.xx.dylib"
$ otool -L /path/to/pythondir/lib/libpython3.xx.dylib
$ cargo clean
$ export PYTHONHOME=/path/to/pythondir
$ export PYTHONPATH=/path/to/pythondir/lib/python3.xx
$ cargo run --example reinforcement-learning --features=pyo3 --package candle-examples -- pg
```

System Info:
cargo 1.85.1 (d73d2caf9 2024-12-31)
rustc 1.85.1 (4eb161250 2025-03-15)
Apple M3 Max (36 GB)
MacOS 15.3.2
python 3.12.7

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.