PyO3 / PyO3/pyo3

Document how to call Python from Rust when the Python code has a nested directory structure

Open
#3,394 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Waiting for response
Dominant language
Rust
Stars
16.2k
Forks
1k
Avg merge
2d 6h
Merged PRs (30d)
66

Description

Please document how to call Python from Rust when the Python code has a nested directory structure.

I'd like to wrap a nested Python module (package?) using repeated calls to PyModule::from_code.
The code has a structure like this:

my_lib
├── a.py
├── b.py
├── foo
│   ├── c.py
│   ├── d.py
│   └── __init__.py
└── __init__.py

The goal is to be able to call into it using PyModule::from_code with code like this:

from my_lib import b
from my_lib.foo import c

def blarg():
  return b.baz() + c.bagel()

Unfortunately, while the guide does describe how to call Python from Rust, and even how to handle dependencies, it doesn't describe how to handle this use-case.

In particular, I'm seeing errors like ModuleNotFoundError: No module named 'my_lib.foo'; 'my_lib' is not a package when adding c.py to my module structure.
I suspect this has something to do with the subtleties of modules, submodule, packages, and subpackages, but I haven't been able to figure it out - I could use some help from someone who understands this!

Thanks!

BTW, I'm in a situation like this and thus cannot simply install and then import my_lib.
Specifically, my code is in three layers: Rust, then Python, then Rust.
The bottom Rust layer defines types, the middle Python layer uses them to expose an API, and the top Rust layer uses the original Rust types with the Python API.
This whole thing needs to live inside a single binary, else I'll have duplicated and incompatible type objects (read through the above link for more info).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Python-from-Rust guide at pyo3.rs/v0.19.2/python_from_rust and the PyModule::from_code entry point. Reproduce the nested my_lib/foo example and investigate the ModuleNotFoundError, including the constraints described in issue #2898. Document a working approach for the three-layer single-binary setup and the nested imports.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.