pybind / pybind/pybind11

Buffer object additions

Open
#1,099 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

If you make something a buffer type, this adds the ability to do:

ob_vals = memoryview(ob)
# or
ob_vals = np.fromarray(ob)

ob_vals[:] = 0
# etc.

There are a standard set of operations that a buffer object will tend to have. [] indexing, length, etc, and they all can be defined with the structure defining the buffer object. Is there a way to set up these standard helper methods? Something like this:

.def_buffer([](MyArray &r) -> py::buffer_info {
     return py::buffer_info(
         r.data(),                         // Pointer to buffer
         {static_cast<ssize_t>(r.size())}, // Diminsions
         {sizeof(float)}                   // Strides (in bytes) for each index
         );
    },
   py::buffer_helpers::getitem<1>(), // 1D
   py::buffer_helpers::setitem<1>() // Etc
)

These helpers could add things like (based on test_sequences_and_iterators):

  • __getitem__, with slicing support, following numpy for 2+ D
  • __setitem__, as above
  • __len__
  • __iter__

Some of these are a bit tricky to add manually, and are fully defined by the buffer_info struct.

Is something like this practical in pybind11? The operators py::self seem to be able to do something like this, and py::pickle, but I didn't see a more general extensible mechanism for adding named methods. (But, since this needs to happen for .def_buffer, it might make sense to add something just for this).

As a side note, the shorter, cleaner, less general versions of py::buffer_info are not mentioned in the documentation, though they are used in the tests.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the .def_buffer entry point and the test_sequences_and_iterators reference in the issue. Determine whether buffer_info can define reusable indexing, assignment, length, and iteration helpers, and review how py::self and py::pickle add named methods. Done should include a settled API direction, coverage for the proposed behavior, and documentation for the shorter py::buffer_info forms.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.