pybind / pybind/pybind11

Best way to get a view of a column of a numpy array?

Open
#2,211 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Suppose I have a function that operates on a 1d array inplace

void a(py::array_t<double> x) { 
  auto a_p = a.mutable_unchecked<1>();
  // do something
}

I now have a 2d array and I would like to implement a function (say, to do multi-threading) that calls a to operate on the 2d array columnwise. What is the correct way to do this?

void b(py::array_t<double> x_2d, int num_threads) {
  boost::asio::thread_pool pool(num_threads);
  for (int i = 0; i < x_2d.shape(1); ++i) {
    boost::asio::post(pool, [&, i] {
      // get x as the ith column of x_2d, how to do it here?
      a(x);
    });
  }
  pool.join();
}

I did something like

x = py::array_t<double>(
  {x_2d.shape(0)}, {x_2d.strides(0)}, x_2d.data(0, i), py::str()
);

But I sometimes got segmentation fault when num_threads is large, not sure why.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the py::array_t construction and the x_2d shape, stride, and data access shown in the issue, then trace how each column is passed to a through boost::asio::thread_pool. Done means identifying a safe column-view approach that remains valid during threaded execution and avoids segmentation faults with larger thread counts.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, numpy
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.