openPMD / openPMD/openPMD-api

HDF5 files are confusing

Open
#1,468 2 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
161
Forks
59
Avg merge
2d 22h
Merged PRs (30d)
7

Description

We looked at an HDF5 file that was generated by openPMD and were very confused that there were so many zeros in the file (see https://github.com/eschnett/CarpetX/issues/152), suspecting a bug in our code. The "solution" was that openPMD extends all datasets to cover the whole domain and fills in the undefined regions with zeros.

I don't think that using the value `0` to indicate an undefined value is convenient. This makes the resulting HDF5 files confusing at best. I recommend using [`H5Pset_fill_value()`](https://docs.hdfgroup.org/hdf5/develop/group___d_c_p_l.html#ga4335bb45b35386daa837b4ff1b9cd4a4) to use a different value (`nan`?) to fill undefined points.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.