openPMD / openPMD/openPMD-api

Reading empty datasets not written via constant record components

Open
#748 0 comments 0 reactions 0 assignees View on GitHub
backend feature request
Dominant language
C++
Stars
161
Forks
59
Avg merge
2d 22h
Merged PRs (30d)
7

Description

The openPMD API uses constant record components to represent empty datasets in a way that is compatible across backends. The openPMD standard does not mandate this, so there might be valid openPMD datasets not written through the openPMD API that represent empty datasets through a regular dataset.

**Is your feature request related to a problem? If so, please describe.**
This exact issue came up in the ADIOS plugin of [PIConGPU](https://github.com/ComputationalRadiationPhysics/picongpu/issues/3256), that talks to ADIOS directly without using the openPMD API.

**Describe the solution you'd like**
While the openPMD API itself should for compatibility reasons stick to constant record components to write empty datasets, it should at least be able to *read* valid datasets that do not use constant record components to represent empty datasets.

Downside: Some backends might behave a little bit weird in the context of empty datasets. ADIOS (versions 1 and 2 alike) will forget dimensionality information on empty datasets, making it impossible to distinguish dataset extents `{0, 5}`, `{5, 0}` and `{0}` for example. A default of `{0}` seems sane enough.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.