pybind / pybind/pybind11

Idea: internal `checked_call` function to minimize error-checking boilerplate

Open
#2,097 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Issue description

The file include/pybind11/pytypes.h contains many instances of error-checking boilerplate in which a Python C API function is called, and the return value is checked. If the return value indicates an error, pybind11 throws an error_already_set exception.

Current checks for errors

Currently, the pybind11 code checks for errors in two ways, based on the Python C API's error reporting.

In some cases, it checks a return value against an expected value of 0, e.g. for PyObject_DelAttr; we could call a "success sentinel".
https://github.com/pybind/pybind11/blob/fb910ae92b4a489e9b31f9ec545bde9d09d6a40e/include/pybind11/pytypes.h#L395-L397

In other cases, it checks a return value against a value like nullptr or -1 indicating an error; we could call this a "failure sentinel".
https://github.com/pybind/pybind11/blob/fb910ae92b4a489e9b31f9ec545bde9d09d6a40e/include/pybind11/pytypes.h#L378-L383

https://github.com/pybind/pybind11/blob/fb910ae92b4a489e9b31f9ec545bde9d09d6a40e/include/pybind11/pytypes.h#L403-L407

Overall there are about 28 cases of this in pytypes.h, and about ~10 in other files, as identified with git grep 'throw error_already_set' include/.

Current non-checks for errors

In other cases, pybind11 fails to check for errors, which has lead to bugs (e.g. #2076).

https://github.com/pybind/pybind11/blob/fb910ae92b4a489e9b31f9ec545bde9d09d6a40e/include/pybind11/pytypes.h#L1198-L1204

https://github.com/pybind/pybind11/blob/fb910ae92b4a489e9b31f9ec545bde9d09d6a40e/include/pybind11/pytypes.h#L1224

https://github.com/pybind/pybind11/blob/fb910ae92b4a489e9b31f9ec545bde9d09d6a40e/include/pybind11/pytypes.h#L1242-L1245

Having a dedicated function that handles the error-checking boilerplate would hopefully lead to more consistent error-checking across the codebase.

Proposal: checked_call function template.

I suggest that we could replace these with a dedicated utility function, so they look like the following:

 inline void delattr(handle obj, handle name) { 
     detail::checked_call<PyObject_DelAttr>(obj.ptr(), name.ptr());
 } 
 inline bool isinstance(handle obj, handle type) { 
     return detail::checked_call<PyObject_IsInstance>(obj.ptr(), type.ptr()); 
 }
 inline object getattr(handle obj, handle name) { 
     PyObject *result = detail::checked_call<PyObject_GetAttr>(obj.ptr(), name.ptr()); 
     return reinterpret_steal<object>(result); 
 } 

Please note that the name of "checked_call" is just a placeholder, and we could bikeshed about the name.

I think this could be implemented by developing traits for the Python C API functions. The checked_call function would forward its arguments, and check the result using traits. If the result indicates an error, error_already_set would be thrown. Overall, the implementation should be ~5 lines, plus the traits classes.

template <typename F, typename... Args>
typename api_traits<F>::return_type checked_call(Args&&... args) {
    const auto result = F(std::forward<Args>(args)...);

    if (api_traits<F>::is_error(result)) {
        throw error_already_set();
    }

    return result;
}

The traits would be implemented using the "success sentinel" and "failure sentinel" models described above.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in include/pybind11/pytypes.h and inspect the existing error checks referenced in the issue; use git grep 'throw error_already_set' include/ to find related cases. Review the unchecked Python C API calls and determine how traits would cover their success and failure sentinels. Done means the utility consistently handles the identified calls without changing their error behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.