Using preallocated buffers in python
- Dominant language
- C++
- Stars
- 161
- Forks
- 59
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 7
Description
**Problem**
Hi! I was often running into the situation where I wanted to do sth like that:
```python
a=np.zeros((2, 2, 1728 , 100), dtype=np.float32)
a = np.ascontiguousarray(a)
a[0, :, :, :] = some_mrc[240:242, :, 100:200]
a[1, :, :, :] = other_mrc[240:242, :, 100:200]
series.flush()
```
So, reading different chunks to different parts of a predefined `numpy` array.
But this fails with: `ValueError: vector::_M_default_append`
There seams to exists an, as far as I know, undocumented workaround, that I just found:
```python
a=np.zeros((2, 2, 1728 , 100), dtype=np.float32)
a = np.ascontiguousarray(a)
some_mrc.load_chunk(a[0, :, :, :], offset=[240,0,100], extent=[2,1728,100])
other_mrc.load_chunk(a[1, :, :, :], offset=[240,0,100], extent=[2,1728,100])
series.flush()
```
As the `MeshRecordComponent.load_chunk` method can take something providing the python buffer interface as an extra argument. But this is not that easy to use as with the slicing syntax and the `[ ]` operator.
**Possible solution**
It would be fantastic if sth like this `a[0, :, :, :] = example_mrc[240:242, :, 100:200]` could work, but I'm not sure how one would change the function execution based on the object on the left-hand side (though `numpy` broadcasting seams to do it somehow).
Alternatively, one could have a new `MeshRecordComponent` method returning an object with a `__get_item__` method, so that we can use syntax like this:
```python
a=np.zeros((2, 2, 1728 , 100), dtype=np.float32)
a = np.ascontiguousarray(a)
some_mrc.get_buffer_loader(buffer=a[0, :, :, :])[240:242, :, 100:200]
other_mrc.get_buffer_loader(buffer=a[1, :, :, :])[240:242, :, 100:200]
series.flush()
```
Or maybe, there exists some class that is exposed in python that can be used to convert the slicing syntax to offset and extent? Sth like: `some_mrc.load_chunk(a[0, :, :, :], *io.offset_and_extent_getter[240:242, :, :100:200])`
I don't know much about how the python bindings work, otherwise I would implement it myself.
Contributor guide
Assessment
This issue has not been assessed yet.