PyO3 / PyO3/pyo3

Borrowed constructor argument can outlive its owner

Open
#6,392 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
16.2k
Forks
1k
Avg merge
2d 6h
Merged PRs (30d)
66

Description

Bug Description

PyO3 keeps borrowed references to keyword arguments while extracting constructor arguments. However, converting one argument can execute Python code through __index__, __str__, or another conversion hook. That code can delete another value from the same kwargs dictionary.

Steps to Reproduce

Minimal Rust reproducer and observed output:

use std::cell::RefCell;
use pyo3::prelude::*;
use pyo3::ffi;
use pyo3::types::{PyDict, PyTuple};

struct Ignored;

impl<'a, 'py> FromPyObject<'a, 'py> for Ignored {
    type Error = PyErr;

    fn extract(obj: Borrowed<'a, 'py, PyAny>) -> Result<Self, Self::Error> {
        let _ = obj.get_type();
        println!("second conversion reached");
        Ok(Self)
    }
}

#[pyclass]
struct Victim;

impl Drop for Victim {
    fn drop(&mut self) { println!("Victim::drop: second finalized"); }
}

#[pyclass(unsendable)]
struct Index { kwargs: RefCell<Option<Py<PyDict>>> }

#[pymethods]
impl Index {
    fn __index__(&self, py: Python<'_>) -> PyResult<i64> {
        println!("Index::__index__: deleting kwargs['second']");
        let kwargs = self.kwargs.borrow_mut().take().unwrap();
        let victim_ptr = {
            let victim = kwargs.bind(py).get_item("second")?.unwrap();
            victim.as_ptr()
        };
        println!("Victim refcount before deletion: {}", unsafe {
            ffi::Py_REFCNT(victim_ptr)
        });
        kwargs.bind(py).del_item("second")?;
        println!("Victim refcount after deletion: {}", unsafe {
            ffi::Py_REFCNT(victim_ptr)
        });
        Ok(7)
    }
}

#[pyclass]
struct C;

#[pymethods]
impl C {
    #[new]
    fn new(first: i64, second: Ignored) -> Self {
        let _ = (first, second);
        println!("constructor reached");
        C
    }
}

fn main() -> PyResult<()> {
    Python::attach(|py| {
        let kwargs = PyDict::new(py);
        let index = Py::new(py, Index {
            kwargs: RefCell::new(Some(kwargs.clone().unbind())),
        })?;
        kwargs.set_item("first", index)?;
        kwargs.set_item("second", Py::new(py, Victim)?)?;
        let _ = py.get_type::<C>().call(PyTuple::empty(py), Some(&kwargs))?;
        Ok(())
    })
}

Output:

Index::__index__: deleting kwargs['second']
Victim refcount before deletion: 1
Victim::drop: second finalized
Victim refcount after deletion: 0
second conversion reached
constructor reached
Backtrace

Your operating system and version

macOS 26.5.2

Your Python version (python --version)

3.14.2

Your Rust version (rustc --version)

1.91.1

Your PyO3 version

0.29.2

How did you install python? Did you use a virtualenv?

brew

Additional Info

I suppose the safest fix is to make argument extraction own each value before running any conversion code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No project file or test is named. Start by running the minimal Rust reproducer and tracing PyO3's constructor argument extraction while conversion hooks mutate kwargs; done when the extracted value remains valid through conversion and the reproducer no longer finalizes the deleted argument prematurely.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.