pybind / pybind/pybind11

[QUESTION] vector of variants as a NumPy array

Open
#2,655 15 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

I'm trying to create a pybind11 interface which returns a tuple of two numpy arrays from C++ to Python. The subfunction lttb inside the lambda expression returns two vectors of type std::vector<double> and std::vector<std::variant<uint32_t, in64_t, double>>.

If I directly return the vectors, the pybind11/stl.h header takes care of the conversion just fine. The creation of the double numpy array also works flawlessly. However, when trying to create a numpy array of std::variant, I get the following error

..\subprojects\pybind11\include\pybind11/numpy.h(1159): error C2338: Attempt to use a non-POD or unimplemented POD type as a numpy dtype

There is a way to register structured types (https://pybind11.readthedocs.io/en/stable/advanced/pycpp/numpy.html#structured-types) in order to prevent this error message, however I only see a way to make this work with a struct not with an std::variant. Does anyone know how to create a numpy array of std::variant? Should not be that hard as the conversion of an std::vector<std::variant<...>> works out of the box. Here's the code.

    .def("lttb",
         [&](ScopeDataElement& data,
             const std::pair<double, double>& limit,
             int64_t masterTick,
             long double samplingTime,
             size_t destinationSize) {
           auto&& [x, y] =
             data.lttb(limit, masterTick, samplingTime, destinationSize);
           using T = std::decay_t<decltype(x)>::value_type;
           using U = std::decay_t<decltype(y)>::value_type;
           auto arrx = py::array_t<T>(x.size(), x.data());
           auto arry = py::array_t<U>(y.size(), y.data());
           return py::make_tuple(arrx, arry);
         })

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the lambda in the issue body and pybind11/numpy.h around line 1159, then compare the linked structured-types documentation. Done means determining whether std::variant can be represented as a NumPy dtype and, if support is appropriate, covering the behavior with a regression test; no existing test file is named.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, numpy, python
Domain
devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.