pybind / pybind/pybind11

[BUG]: can not access the target (std::function) of python functions

Open
#3,908 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
Problem description

Motivation:

For callback handling I would like to be able to compare callbacks / functions passed from python via pybind to a c++ implemented comparison function.

  • callbacks could be either "pure" python functions or pybind-wrapped c++ functions
  • comparisons should be possible between all combinations of function types (python vs python, python vs c++, c++ vs c++)

Solution:

To compare 2 std::function instance one needs to actually compare their targets (=function pointer):

bool equal(std::function<void()> f1, std::function<void()> f2) {
	if (f1.target_type() != f2.target_type()){
		std::cout << "f1 and f2 of different types: " << f1.target_type().name() << " vs " << f2.target_type().name() << std::endl;
		return false;
	}else{
		std::cout << "f1 and f2 of same type: " << f1.target_type().name() << std::endl;
	}

	auto **p_f1 = f1.target<void(*)()>();
	auto **p_f2 = f2.target<void(*)()>();
	if(p_f1==0){
		std::cout << "f1 can not be cast to : void(*)()" << std::endl;
		return false;
	}
	if(p_f2==0){
		std::cout << "f2 can not be cast to : void(*)()" << std::endl;
		return false;
	}

	if (*p_f1 != *p_f2)
		return false;

	return true;
}

PYBIND11_MODULE(example, m) {
	m.def("equal", &equal, "A function that compares two functions");
}

Problem:

When passing two different or twice the same std::function instance to the function above, the comparison works correctly as expected.
But when passing a "pure" python function, it's target can not be cast to the correct function type/pointer (= void(*)()). The target will always point to address 0.

Reproducible example code
# test.py (test script)

from example import equal, cpp_callback1, cpp_callback2

def py_callback1():
    print('py_callback1 called')

def py_callback2():
    print('py_callback2 called')

print(equal(cpp_callback1, cpp_callback1))
print(equal(cpp_callback1, cpp_callback2))
print(equal(py_callback1, py_callback1))
print(equal(py_callback1, py_callback2))
// example.cpp (pybind code)
#include <functional>
#include <iostream>
#include <pybind11/pybind11.h>
#include <pybind11/functional.h>

namespace py = pybind11;

bool equal(std::function<void()> f1, std::function<void()> f2) {
	if (f1.target_type() != f2.target_type()){
		std::cout << "f1 and f2 of different types: " << f1.target_type().name() << " vs " << f2.target_type().name() << std::endl;
		return false;
	}else{
		std::cout << "f1 and f2 of same type: " << f1.target_type().name() << std::endl;
	}

	auto **p_f1 = f1.target<void(*)()>();
	auto **p_f2 = f2.target<void(*)()>();
	if(p_f1==0){
		std::cout << "f1 can not be cast to : void(*)()" << std::endl;
		return false;
	}
	if(p_f2==0){
		std::cout << "f2 can not be cast to : void(*)()" << std::endl;
		return false;
	}

	if (*p_f1 != *p_f2)
		return false;

	return true;
}

void cpp_callback1() {
	std::cout << "cpp_callback1 called" << std::endl;
}
void cpp_callback2() {
	std::cout << "cpp_callback2 called" << std::endl;
}

PYBIND11_MODULE(example, m) {
	m.def("equal", &equal, "A function that compares two functions");

	m.def("cpp_callback1", &cpp_callback1, "A test callback function");
	m.def("cpp_callback2", &cpp_callback2, "An other callback function");
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the callback comparisons from test.py using example.cpp, then inspect pybind11/functional.h to trace how Python callables become std::function targets. Check the Python/Python and Python/C++ combinations described in the issue; done means the requested comparisons produce the expected results.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.