HDFGroup / HDFGroup/hdf5

✨ [Feature Request] String-Based Configuration Interface for Filters

Open
#6,153 43 comments 2 reactions 1 assignee Claimed by @brtnfld View on GitHub
Component - C Library HDFG-internal
Dominant language
C
Stars
988
Forks
355
Avg merge
4d 2h
Merged PRs (30d)
12

Description

### 1. Problem Description
Currently, the generic interface for configuring HDF5 filters relies exclusively on `cd_values`, an array of `unsigned int`. While this works for simple integer parameters, it creates significant friction for modern filters (like **H5Z-ZFP**) that require floating-point parameters (rates, precision, thresholds).

**Pain Points:**
* **Type Punning:** Users must manually bit-cast `double` or `float` values into integer arrays. This is error-prone, endian-sensitive, and difficult to implement in high-level languages (Python, Java, Matlab) without dedicated wrappers.
* **CLI Usability:** Tools like `h5repack` require users to pass "magic numbers" (e.g., `-f UD=32013,0,4,1,0,0,1074921472`) instead of human-readable settings.
* **Opaque Configuration:** It is difficult to introspect a file's creation properties and understand what compression settings were used (e.g., `cd_values[2]` is meaningless compared to `"rate=4.0"`).

### 2. Proposed Solution
Introduce a mechanism to configure filters using a string-based key-value interface, mirroring modern capabilities recently added for Virtual File Drivers (VFDs) and VOL connectors.

The underlying storage format (the `cd_values` array stored in the object header) **should remain unchanged**. This feature would simply be a usability layer that allows the filter plugin to parse a string and populate the `cd_values` array itself.

### 2. Example Usage

#### C API Comparison
**Current (Manual Type Punning):**
```c
// User must know internal layout and bit-casting
unsigned int cd_vals[4];
double rate = 3.5;
cd_vals[0] = H5Z_ZFP_MODE_RATE;
memcpy(&cd_vals[2], &rate, sizeof(double)); // Complex & unsafe
H5Pset_filter(plist, H5Z_FILTER_ZFP, flags, 4, cd_vals);
```

**Proposed:**
```c
// User specifies intent; Plugin handles the bits
H5Pset_filter_config(plist, H5Z_FILTER_ZFP, "mode=rate; rate=3.5");
```

#### CLI Impact (`h5repack`)
This would allow `h5repack` to support a generic string flag, removing the need for external helper scripts like `print_h5repack_farg`.

**Current:**
```bash
./print_h5repack_farg rate=3.5
# Output: -f UD=32013,0,4,1,0,0,1074528256
h5repack -f UD=32013,0,4,1,0,0,1074528256 in.h5 out.h5
```

**Proposed:**
```bash
h5repack -f ZFP="mode=rate; rate=3.5" in.h5 out.h5
```

### 3. References
* Related Issue: [llnl/H5Z-ZFP#159](https://github.com/llnl/H5Z-ZFP/issues/159) - Discusses the complexity of generic usage.
* Precedent: HDF5 1.14 VFD configuration strings (`H5Pset_driver_by_name` configuration argument).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.