PyO3 / PyO3/pyo3

`PyDict::iter` don't align with python semantics

Open
#6,222 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
16.2k
Forks
1k
Avg merge
2d 6h
Merged PRs (30d)
66

Description

In python, dict[K, V]::__iter__ return an Iterator[K], i.e over the keys.

Pyo3 is an Iterator over keys/values pairs, which is very surprising, especially when an item method exist.

This Iterator should be either a dedicated method (iter_items), or moved to the PyDictItems type.

In a broader scope, better views object support would be welcome.

There's not an easy way to get views objects from a dict for example, and the existing methods encourage the innefficient pattern of iter(list(dict.keys()))

So I added that trait for my work:

#[allow(unused)]
pub trait IntoPyMappingView<'py> {
    fn items_view(&self) -> Bound<'py, PyDictItems>;
    fn keys_view(&self) -> Bound<'py, PyDictKeys>;
    fn values_view(&self) -> Bound<'py, PyDictValues>;
}
impl<'py> IntoPyMappingView<'py> for Bound<'py, PyDict> {
    fn items_view(&self) -> Bound<'py, PyDictItems> {
        self.call_method0(intern!(self.py(), "items"))
            .unwrap()
            .pipe(|x| unsafe { x.cast_into_unchecked::<PyDictItems>() })
    }
    fn keys_view(&self) -> Bound<'py, PyDictKeys> {
        self.call_method0(intern!(self.py(), "keys"))
            .unwrap()
            .pipe(|x| unsafe { x.cast_into_unchecked::<PyDictKeys>() })
    }
    fn values_view(&self) -> Bound<'py, PyDictValues> {
        self.call_method0(intern!(self.py(), "values"))
            .unwrap()
            .pipe(|x| unsafe { x.cast_into_unchecked::<PyDictValues>() })
    }
}

But still, PyDictValues is treated as opaque, so at call site I still have to add the following if I want to actually iterate over those values:

            .as_any()
            .try_iter()
            .unwrap()

UNLESS there's a particular specific reason to not support this, but this is mentionned nowhere in the documentation, which would be nice, since all mappings/dict methods related to view eagerly copies them to PyList.
I know that this is what the C API actually give for the PyDict_Keys for example, but this is mainly due to retro-compatibility with older python versions who did not have such view objects.
Since I do believe Pyo3 is not constrained by this (at least to the same extent), improving the related API's would be very welcome.

By "broader scope", I mean a lot of functionnalities related to python builtins, that I already added here, and there for example.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from the PyDict::iter entry point and the PyDictItems, PyDictKeys, and PyDictValues types. Compare their behavior with Python dict iteration and view semantics, then determine the intended API for key iteration and view access. Done should include an agreed scope, implementation, and tests or documentation covering the resulting behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.