pybind / pybind/pybind11

Proposal: introduce a call guard factory

Open
#2,183 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Hi. Working on some glue code between Python and V8. I just ported quite non-trivial boost::python project under pybind11. And my mission was successful. I'm still amazed how much effort was put into this library. Kudos!

The problem

I have pretty heavy logging going on in the dev mode. I wanted to automatically track all calls crossing Python->C++ boundary. To keep the task simple, let's say I want to log the invoked python method name with each call. (Later I want to do more sophisticated pre-call/post-call logic)

I didn't really understand internal pre/post-call code. I wanted to use py::call_guard which is documented but I found it rather limiting. The problem is that it has to be stateless to be default-constructible and I don't have an easy way how to hook all "def-like" places. Of course I could wrap every method used in .def* with a lambda or do something there but how to best do this in an automated central way?

I ended up with a working solution, but I don't like it:
https://github.com/darwin/naga/blob/8d0fee2f1207481f78cd355dc94796f01553d911/src/PybindNagaClass.h
Here is an example where I use the wrapper class instead of py::class_:
https://github.com/darwin/naga/blob/8d0fee2f1207481f78cd355dc94796f01553d911/src/PythonExpose.cpp#L111

The code looks the same like using vanilla py::class_, which was the goal.

More flexible solution

I guess that the limitation of call_guard comes from the support for composition of multiple guards. I don't care much about the composition, because I can easily compose more complex guards myself. What I need is some captured state.

Ideally I would like to pass a lambda (with captures) or just some callable object (with state) which would act as a factory for guards.

The call_impl code here could be something like:

    template <typename Return, typename Func, size_t... Is, typename GuardFactory>
    Return call_impl(Func &&f, index_sequence<Is...>, GuardFactory&& gf) && {
        auto guard = gf();
        return std::forward<Func>(f)(cast_op<Args>(std::move(std::get<Is>(argcasters)))...);
    }

Thank you for any suggestions and considering this. I could also attempt a proof-of-concept implementation with some guidance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with call_impl in include/pybind11/cast.h and the documented py::call_guard behavior. Compare the linked PybindNagaClass.h wrapper and PythonExpose.cpp usage to understand the requested central hook. Done means agreeing on and demonstrating a guard-factory design that supports captured state for pre-call and post-call logic.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.