[QUESTION] How to efficiently stop iteration?
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 18k
- Forks
- 2.3k
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 10
Description
I defined an iterable class in pybind11 by creating a __getitem__ method which throws a std::out_of_range exception if the index is out of range. While this successfully stops the iteration in python (without raising a python exception), it appears that stopping the iteration takes about 100us vs 1us for a native implementation, so ~100x slower. I tried the following things:
A. PyErr_SetString(PyExc_StopIteration, "..."); throw error_already_set();: takes ~25us, so ~4x faster than std::out_of_range but still ~25x slower than the native version.
B. PyErr_SetString(PyExc_StopIteration, "..."); and then propagating the error manually, without throwing a C++ exception. This takes ~5us, so ~20x faster than std::out_of_range and "only" ~5x slower than the native version.
However, option B requires returning a NULL PyObject to python, which the dispatcher doesn't like- it triggers the error here: https://github.com/pybind/pybind11/blob/master/include/pybind11/pybind11.h#L924-L928. In order to run option 2, I have to patch those lines to check PyErr_Occurred() and only set an error if none is already set.
so, my questions:
- Is there any equally or more efficient way to define an iterable object in C++ using pybind11 than
__getitem__and option B above? - If the answer to that is "no", can
cpp_function::dispatcherbe patched to allow setting a python error directly, without throwing a C++ exception?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the cpp_function::dispatcher code in include/pybind11/pybind11.h around lines 924-928, then compare the reported __getitem__ approaches and their timings. Determine whether direct propagation of an existing Python error can be supported without a C++ exception, and verify that iteration stops without replacing the pending error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- devtools
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100