pybind / pybind/pybind11

[QUESTION] Throwing error messages in an overloaded custom caster

Open
#3,146 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

I created a custom caster that passes a reference of a cupy array from python to C++. I created a container to store the details of the cupy array.

On the C++ side, the container is like this :

#pragma once

template<typename TData>
class Custom_Cupy_Ref {
  public :
    TData * ptr;
    std::size_t size;
    std::string dtype;
};

On the python side, the container is like this :

class Custom_Cupy_Ref:
    def __init__(self, ptr, size, dtype):
        self.ptr = ptr
        self.size = size
        self.dtype = dtype

I want to pass a cupy reference class as argument and use it in C++ like the following code :

b = cupy_ref.Custom_Cupy_Ref(ptr = a.data.ptr, size = a.size, dtype = str(a.dtype))
    res = cupy_binding.Foo(b)

The binding code is like this :

m.def("foo", [](Custom_Cupy_Ref input){ // .. do stuff with input});

Therefore I wrote a type caster :

template<typename TData> std::string cupy_ref_get_dtype(){ return "C++ type not implemented";}

template<> std::string cupy_ref_get_dtype<float>(){ return "float32";}
template<> std::string cupy_ref_get_dtype<double>(){ return "float64";}
template<> std::string cupy_ref_get_dtype<std::uint16_t>(){ return "uint16";}
template<> std::string cupy_ref_get_dtype<std::uint32_t>(){ return "uint32";}
template<> std::string cupy_ref_get_dtype<std::complex<float>>(){ return "complex64";}
template<> std::string cupy_ref_get_dtype<std::complex<double>>(){ return "complex128";}

namespace pybind11 { namespace detail {
    template <typename T> struct type_caster<Custom_Cupy_Ref<T>> 
    {
    public:
        PYBIND11_TYPE_CASTER(Custom_Cupy_Ref<T>, _("cupy_ref.Custom_Cupy_Ref"));

        // python -> C++
        bool load(handle src, bool)
        {
            if(!hasattr(src, "ptr") && !hasattr(src, "size"))
            {
                 return false;
            }
            
            //check if dtype matches the expected C++ type
            if(src.attr("dtype").cast<std::string>() != cupy_ref_get_dtype<T>()){
			std::ostringstream oss;
			oss << "Cupy Ref type missmatch\n";
			oss << "  Python type: " << src.attr("dtype").cast<std::string>() << "\n";
			oss << "  Expected Python type: " << cupy_ref_get_dtype<T>() << "\n";
			std::cerr << oss.str();
			return false;
            }


            value.ptr = reinterpret_cast<T *>(src.attr("ptr").cast<size_t>());
            value.size = src.attr("size").cast<size_t>();
            value.dtype = src.attr("dtype").cast<string>();

            return true;
        }
  }

One thing that the code above do is to check whether the dtype of cupy array matches the expected type of the C++ class of Custom_Cupy_Ref.

If the python binging is not overloaded, everything is fine. But if it is overloaded, it also works but I got a lot of output which should not appear.

m.def("Foo", &Foo<uint16_t>);
m.def("Foo", &Foo<uint32_t>);
m.def("Foo", &Foo<float>);
m.def("Foo", &Foo<double>);
m.def("Foo", &Foo<complex<float>>);
m.def("Foo", &Foo<complex<double>>);
Cupy Ref type missmatch
  Python type: complex128
  Expected Python type: uint16
Cupy Ref type missmatch
  Python type: complex128
  Expected Python type: uint32
Cupy Ref type missmatch
  Python type: complex128
  Expected Python type: float32
Cupy Ref type missmatch
  Python type: complex128
  Expected Python type: float64
Cupy Ref type missmatch
  Python type: complex128
  Expected Python type: complex64

Is there a solution so the error messages is not appeared for each type everytime an overloaded binding is called?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the custom pybind11 type_caster<Custom_Cupy_Ref>::load implementation and the overloaded Foo bindings shown in the issue. Reproduce the complex128 call and trace how failed dtype checks are handled during overload resolution. Done means overload probing no longer emits mismatch diagnostics for candidates that are not selected.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.