Consider making it possible to borrow `PyRef` in `#[pyclass]`
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 16.2k
- Forks
- 1k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 66
Description
At the moment, as #1088 notes (and also #1085), it's difficult to expose things like iterators to Python because of the restriction that #[pyclass] cannot take a lifetime.
This restriction obviously makes sense because Rust-managed data cannot be borrowed and then exposed to Python; Python will allow this data to live arbitrarily long.
However, data that's already owned by Python, e.g. in a PyCell<T>, can be safely "borrowed" and stored in another #[pyclass]. We do this already, but only for the lifetime-free case Py<T>.
It would be incredibly powerful if PyRef<'a, T> (or some similar construct) was able to be stored inside another #[pyclass]. This would make make wrapping arbitrary iterators, for example, much easier.
This playground shows that with some carefully-placed transmutes it's possible to hack the type system to at least achieve the desired affect: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=041f9a6f5dfa3a5fc59ece7105cc4dfd
That code is a wildly unsafe sketch that doesn't actually use the PyO3 types, so please don't try to use it as-is. You're responsible for all segfaults if you do 😄. If you're finding yourself in need of this feature, I can offer mentoring to help design how PyO3 could make such a trick safe and ergonomic.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the discussion in issues #1088 and #1085, then study the PyRef, PyCell, and #[pyclass] concepts described here. The linked Rust playground illustrates the lifetime challenge but is explicitly unsafe and not an implementation. Done means defining and documenting a safe, ergonomic way to store borrowed Python-owned data in another #[pyclass].
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100