Sharing pyclasses between multiple Rust packages

I've seen several questions about re-using #[pyclass] types between multiple Rust packages, for example on gitter, stack overflow and most recently in #1418.

This issue is indended to be a place for the discussion to centralize regarding understanding the difficulties in this and to brainstorm solutions / API designs which we can add to PyO3.

Below is my write-up of the issue. Note that I haven't tried to any of this myself; this is just where I think the difficulties are. I could be totally wrong - please use this thread to share evidence, ask questions, and test ideas.

Let's call package A the "original" package, which defines a #[pyclass] MyClass. Package B is a child crate which makes use of use a::MyClass in Rust code, to re-use the pyclass.

The key issue is that #[pyclass] stores the pyclass type object in static storage. This means that (if Rust's usual rlib linkage is used) packages A and B will have their own copies of the MyClass type object, and Python will think that they're actually different types coming from the two packages.

I think the core of the solution to this is to make B link against the package A C shared library - the same .so file which Python usess to load package A. Then B should be able to re-use the pyclass from A.

This comes with a few steps:

• A will need to export additional symbols which B can use. This probably includes at a minimum the MyClass static type object, and probably functions for converting PyAny <-> MyClass.
• B will need to use A as a "C" dependency which it links to using an extern "C" block, where it imports symbols from A.
• I'm unsure if B will be able to use a::MyClass; at all - it may need to have a duplicate definition which uses the exported symbols internally.

Looking at that, it might mean that we'd eventually have pyclass_export! and pyclass_import! macros in PyO3 to help with this.

The above is all a poorly-worded brain dump of what might be a solution. I'm really not sure - anyone who is trying to do this, please comment, ask questions, provide error logs, and we can figure this out together.

Start by reading this issue alongside #1418 and the linked Gitter and Stack Overflow discussions, then reproduce the multi-package #[pyclass] case described for packages A and B. Document the required symbols and linkage behavior, and define what a workable export/import API would need to demonstrate.