openPMD / openPMD/openPMD-api

Parameter Read/Write: Data Policy

Open
#470 2 comments 0 reactions 0 assignees View on GitHub
backend help wanted refactoring
Dominant language
C++
Stars
161
Forks
59
Avg merge
2d 22h
Merged PRs (30d)
7

Description

It would probably be useful if we added a trait / policy or runtime converter to the [`Parameter< Operation::READ_DATASET >`](https://github.com/openPMD/openPMD-api/blob/0.7.1-alpha/include/openPMD/IO/IOTask.hpp#L321-L340) (and write) data members of shared pointers in order to share more complex data objects.

For example returning to python a `py::array`, that the user might not handle, might result in it getting garbage collected before our `flush()` can be called, which in turn causes in invalid memory access in the backend. We could keep that `py::array` alive until `flush()` as in C++ by returning a `shared_ptr< py::array >`.

Overloads for [`RecordComponent::loadChunk`](https://github.com/openPMD/openPMD-api/blob/0.7.1-alpha/include/openPMD/RecordComponent.hpp#L276) are already okay and well designed, but we cannot share yet an object that needs a more indirect data access pattern than a shared_ptr of raw memory, since we access it right away:
https://github.com/openPMD/openPMD-api/blob/0.7.1-alpha/src/IO/HDF5/HDF5IOHandler.cpp#L1017

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.