[BUG]: Recursive dispatch fails to find python function override.

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

Problem description

Consider a simple C++ visitor pattern:

class Data{};
using DataVisitor = std::function<void (const Data&)>;

class Adder {
public:
    // interface to add two data objects, the result is passes to data visitor
    virtual void operator()(const Data& first, const Data& second, const DataVisitor& visitor) const = 0;
};

void add(const Data& first, const Data& second,  const Adder& adder, const DataVisitor& visitor) {
    adder(first, second, visitor);
}

void add3(const Data& first, const Data& second, const Data& third,  const Adder& adder, const DataVisitor& visitor) {
    adder(first, second, [&] (const Data& first_plus_second) {
        adder(first_plus_second, third, visitor);
    });
}

In this pattern, Data and Adder are abstract classes which can be used in two algorithms add and add3 which, as the name suggest, add two or three objects of the type Data and send the result to a visitor. This pattern is intended to eliminate allocating memory for Data pointer in case we need to store temporary results, e.g., see add3 algorithm.

Next, we define the corresponding python bindings:

class PyAdder : public Adder {
public:
    void operator()(const Data& first, const Data& second, const DataVisitor& visitor) const override {
        PYBIND11_OVERRIDE_PURE_NAME(void, Adder, "__call__", operator(), first, second, visitor);
    }
};

class PyData : public Data {
public:
    PyData() = default;
};

PYBIND11_MODULE(module, m) {
    pybind11::class_<Adder, PyAdder>(m, "Adder")
        .def(pybind11::init<>())
        .def("__call__", &Adder::operator());

    m.def("add", &add);
    m.def("add3", &add3);

    pybind11::class_<Data, PyData>(m, "Data")
        .def(pybind11::init<>());
}

Finally, let's test it in Python. Firs,t some boilerplate:

class Adder(module.Adder):
    def __call__(self, first, second, visitor):
        visitor(Data(first.value + second.value))

class Data(module.Data):
    def __init__(self, value):
        super().__init__()
        self.value = value

def print_result(data):
        print(data.value)

Testing function add yields the expected result

module.add(Data(1), Data(2), Adder(), print_result) # prints 3

However, testing add3 causes failure

module.add3(Data(1), Data(2), Data(3), Adder(), print_result) # raises RuntimeError

The error reads

Traceback (most recent call last):
  File "*****/test_add.py", line 39, in test_add3_int
    module.add3(Data(1), Data(2), Data(3), Adder(), print_result)
  File "*****/test_add.py", line 8, in __call__
    visitor(Data(first.value + second.value))
RuntimeError: Tried to call pure virtual function "Adder::__call__"

Chaising the problem through pybind11 sources, I found a possible culprit. In function get_type_override, we check if dispatch code is invoked from overridden function.

    /* Don't call dispatch code if invoked from overridden function.
       Unfortunately this doesn't work on PyPy. */
#if !defined(PYPY_VERSION)
    PyFrameObject *frame = PyThreadState_Get()->frame;
    if (frame != nullptr && (std::string) str(frame->f_code->co_name) == name
        && frame->f_code->co_argcount > 0) {
        PyFrame_FastToLocals(frame);
        PyObject *self_caller = dict_getitem(
            frame->f_locals, PyTuple_GET_ITEM(frame->f_code->co_varnames, 0));
        if (self_caller == self.ptr())
            return function();
    }
#else

It returns an empty function if the second override is called from the same frame as the first override. Commenting this check renders a working example. I consider this as a bug, but please let consider proving me wrong.

My questions are:

• In which case it is important to prevent the second dispatch from overridden function?
• Is there a good way to fix things related to Visitor (recursive) or Double Dispatch patterns?

The link for reproducible and extended examples: https://bitbucket.org/yershov/recursive-dispatch/src/main/

Reproducible example code

https://bitbucket.org/yershov/recursive-dispatch/src/main/

Start with get_type_override and the recursive-dispatch example linked in the issue. Reproduce the add3 failure and inspect the frame check that returns an empty function during nested overrides. Done means establishing the intended dispatch behavior and adding coverage showing that the recursive visitor example no longer raises the pure-virtual-function error.