PyO3 / PyO3/pyo3

`__anext__` + `experimental-async` support

Open
#5,773 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
16.2k
Forks
1k
Avg merge
2d 6h
Merged PRs (30d)
66

Description

Bug Description

In starting to use experimental-async I noticed that __anext__ isn't allowed to be an async function.

Steps to Reproduce

REPO: https://github.com/jessekrubin/pyo3-anext-repro

copy-pasta-ed some of the experimental-async code to write das test.

use futures::channel::oneshot;
use pyo3::prelude::*;
use std::{sync::atomic::Ordering, thread, time::Duration};

fn handle_windows(test: &str) -> String {
    let set_event_loop_policy = r#"
    import asyncio, sys
    if sys.platform == "win32":
        asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
    "#;
    pyo3::impl_::unindent::unindent(set_event_loop_policy) + &pyo3::impl_::unindent::unindent(test)
}

#[pyfunction]
async fn sleep(seconds: f64) -> usize {
    let (tx, rx) = oneshot::channel();
    thread::spawn(move || {
        thread::sleep(Duration::from_secs_f64(seconds));
        tx.send(42).unwrap();
    });
    rx.await.unwrap()
}

#[pyclass]
struct AsyncRange {
    count: std::sync::atomic::AtomicUsize,
    max: usize,
}

#[pymethods]
impl AsyncRange {
    #[new]
    fn new(max: usize) -> Self {
        Self {
            count: std::sync::atomic::AtomicUsize::new(0),
            max,
        }
    }

    fn __aiter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> {
        slf
    }

    async fn __anext__(&self) -> Option<usize> {
        let current = self.count.fetch_add(1, Ordering::SeqCst);
        if current < self.max {
            // simulate async work
            sleep(0.001).await;
            Some(current + 1)
        } else {
            None
        }
    }
}

#[test]
fn async_anext() {
    Python::attach(|py| {
        let test = r#"
async def main():
    results = [value async for value in AsyncRange(3)]
    assert results == [1, 2, 3]
asyncio.run(main())
"#;
        let globals = pyo3::types::PyDict::new(py);
        globals
            .set_item("AsyncRange", py.get_type::<AsyncRange>())
            .unwrap();
        py.run(
            &std::ffi::CString::new(pyo3::impl_::unindent::unindent(&handle_windows(test))).unwrap(),
            Some(&globals),
            None,
        )
        .unwrap();
    });
}
Backtrace
jesse@puter[[pyo3-async-anext]]$ cargo test
   Compiling pyo3-build-config v0.28.0
   Compiling pyo3-macros-backend v0.28.0
   Compiling pyo3-ffi v0.28.0
   Compiling pyo3 v0.28.0
   Compiling pyo3-macros v0.28.0
   Compiling repro v0.1.0 (/home/jesse/pyo3-async-anext)
error[E0277]: the trait bound `impl futures::Future<Output = Option<usize>>: IntoPyCallbackOutput<'_, _>` is not satisfied
   --> src/main.rs:30:1
    |
 30 | #[pymethods]
    | ^^^^^^^^^^^^ the trait `IntoPyCallbackOutput<'_, _>` is not implemented for `impl futures::Future<Output = Option<usize>>`
    |
    = help: the following other types implement trait `IntoPyCallbackOutput<'py, Target>`:
              `()` implements `IntoPyCallbackOutput<'_, ()>`
              `()` implements `IntoPyCallbackOutput<'_, i32>`
              `*mut PyObject` implements `IntoPyCallbackOutput<'_, *mut PyObject>`
              `HashCallbackOutput` implements `IntoPyCallbackOutput<'_, isize>`
              `Result<T, E>` implements `IntoPyCallbackOutput<'py, U>`
              `bool` implements `IntoPyCallbackOutput<'_, bool>`
              `bool` implements `IntoPyCallbackOutput<'_, i32>`
              `usize` implements `IntoPyCallbackOutput<'_, isize>`
              `usize` implements `IntoPyCallbackOutput<'_, usize>`
note: required by a bound in `AsyncIterBaseTag::convert`
   --> /home/jesse/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pyo3-0.28.0/src/impl_/pymethods.rs:559:16
    |
557 |     pub fn convert<'py, Value, Target>(self, py: Python<'py>, value: Value) -> PyResult<Target>
    |            ------- required by a bound in this associated function
558 |     where
559 |         Value: IntoPyCallbackOutput<'py, Target>,
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `AsyncIterBaseTag::convert`
    = note: this error originates in the attribute macro `pymethods` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0277`.
error: could not compile `repro` (bin "repro" test) due to 1 previous error
Your operating system and version

windows11/rhel8

Your Python version (python --version)

python313

Your Rust version (rustc --version)

1.92.0

Your PyO3 version

0.28.0

How did you install python? Did you use a virtualenv?

uv

Additional Info

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the linked pyo3-anext-repro example and inspect the #[pymethods] expansion around AsyncIterBaseTag::convert, identified in the backtrace. Done means an async anext method is accepted under experimental-async and the provided async_anext test completes with [1, 2, 3].

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.