Simple slicing for chunked data
- Dominant language
- C++
- Stars
- 161
- Forks
- 59
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 7
Description
I just learned the long way that my naive approach to slice data (or even read full datasets) like
```python
mydata = it.meshes['cons_cons_density_lvl1'][opmd.Record_Component.SCALAR][:,:,10]
series.flush()
do_work_with_mydata()
```
Does not play nicely with chunked data (ADIOS2/mp5 output).
I saw the the openpmd-viewer introduces quite a bit of logic to load slice data https://github.com/openPMD/openPMD-viewer/blob/6eccb608893d2c9b8d158d950c3f0451898a80f6/openpmd_viewer/openpmd_timeseries/data_reader/io_reader/utilities.py#L88
Is there a simpler approach?
I naively would have expected that there's sth like `load_chunk*s*` (potentially hidden inside the `load_chunk` call) that would load the data from all chunks that are part of the dataset.
A related question to the approach above also pertains to performance: The viewer example load each chunk individually by calling many flushes. Is there a better(transparent way) around it (again ideally hidden behind a `load_chunk*s*` call)?
Contributor guide
Assessment
This issue has not been assessed yet.