openPMD / openPMD/openPMD-api

HDF5: Remove Support for Old Versions

Open
#1,514 6 comments 0 reactions 1 assignee Claimed by @ax3l View on GitHub
backend: HDF5 MPI
Dominant language
C++
Stars
161
Forks
59
Avg merge
2d 22h
Merged PRs (30d)
7

Description

If we bump up our support to newer HDF5 versions, then we can optimize meta-data features & performance:

The call `H5Pset_libver_bounds` will set the guaranteed compatibility with older readers. The less old versions are supported, the more performant we can be.

Cool features in Quincy Koziol's ATPESC23 or HUG23 slides (to be linked):
- size of metadata structures `H5Pset_istore_k`, `H5Pset_sym_k`, etc.
- metadata aggregation in larger blocks `H5Pset_meta_block_size`
- Align objects in the file `H5Pset_alignment`
- Metadata cache size
- Page allocation and page buffering (see: File Space management)

We got to check, but can probably dump everything before 1.12 because our users & readers use recent pip/conda packages that mostly ship HDF5 1.14 already.

cc @jeanbez already increased quite a few of these buffers, but I was unaware of `H5Pset_libver_bounds` performance implications so far.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.