PyO3 / PyO3/pyo3

`Python::allow_threads` is unsound in the presence of `scoped-tls`.

Open
#3,640 43 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Unsound
Dominant language
Rust
Stars
16.2k
Forks
1k
Avg merge
2d 6h
Merged PRs (30d)
66

Description

In analogy to

  • #2141

we can smuggle Ungil data across Python::allow_threads using scoped-tls, too.

use pyo3::prelude::*;
use pyo3::types::PyString;
use scoped_tls::scoped_thread_local;

fn main() {
    Python::with_gil(|py| {
        let string = PyString::new(py, "foo");

        scoped_thread_local!(static WRAPPED: PyString);

        WRAPPED.set(string, || {
            py.allow_threads(|| {
                WRAPPED.with(|smuggled: &PyString| {
                    println!("{:?}", smuggled);
                });
            });
        });
    });
}

(results in segfault in my test)

Unlike #2141, this issue is virtually unsolvable, i.e. even the auto trait approach with the feature="nightly" enabled cannot catch this at all. There’s no property in the callback to allow_threads that can catch this. Really, the callback doesn’t even capture anything at all:

use pyo3::prelude::*;
use pyo3::types::PyString;
use scoped_tls::scoped_thread_local;

scoped_thread_local!(static WRAPPED: PyString);

fn callback() {
    WRAPPED.with(|smuggled: &PyString| {
        println!("{:?}", smuggled);
    });
}

fn main() {
    Python::with_gil(|py| {
        let string = PyString::new(py, "foo");

        WRAPPED.set(string, || {
            py.allow_threads(callback); // callback is an ordinary `fn() -> ()` item.
        });
    });
}

Again, there’s a “whose fault” question to be asked, whether the fact that the standard library’s thread-locals require T: 'static means there’s any guarantees that non-'static data isn’t allowed to be “smuggled” through thread-local storage. In my view, if you look at what kind of things it offers, scoped-tls seems even less unreasonable to be called “sound”, compared to sync_wrapper. Yet the consequences for Ungil are more detrimental.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Python::allow_threads and the scoped_thread_local examples in the issue, then review the related discussion in #2141. The issue names no files or tests and does not identify a viable change; done would require an agreed resolution for the unsoundness and evidence that the demonstrated smuggling case is addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
backend, security
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.