HDFGroup / HDFGroup/hdf5

filters: `can_apply` does nothing for optional filters

Open
#6,161 1 comment 0 reactions 1 assignee Claimed by @fortnern View on GitHub
Component - C Library HDFG-internal
Dominant language
C
Stars
988
Forks
355
Avg merge
4d 2h
Merged PRs (30d)
12

Description

From the [documentation of H5Zregister](https://support.hdfgroup.org/documentation/hdf5/latest/group___h5_z.html#title6):

> If this filter is enabled through [H5Pset_filter()](https://support.hdfgroup.org/documentation/hdf5/latest/group___o_c_p_l.html#gae7ac4c110e9ec3fcf38966928b96cda3) as optional and the can apply function returns 0, the library will skip the filter in the filter pipeline.

The above does not seem to be implemented.
The current behaviour is that
1. `H5Pset_filter(..., H5Z_FLAG_MANDATORY, ...)` later causes the call to `H5Dcreate2` to execute `can_apply` and then fail if it returned 0, as expected.
2. `H5Pset_filter(..., H5Z_FLAG_OPTIONAL, ...)` later causes
a. the call to `H5Dcreate2` to execute `can_apply` and, even after it returns 0, to execute `set_local`.
b. the call to `H5DWrite` / `H5DRead` to invoke the `filter` callback for every chunk, even if `can_apply` earlier returned 0.

This behaviour can be seen [here](https://github.com/HDFGroup/hdf5/blob/476c60961c4148c31e738cd1e3e9a253c6917cf8/src/H5Z.c#L799-L818), where the return value of 0 is disregarded.

## Reproducer
#6162

## Workaround
A filter developer could change `set_local` to internally rerun `can_apply`. if it returns 0, `set_local` would set a guard value in `cd_values` and return 0. The `filter` callback would then need to perform an early check for the guard in cd_values.

## Context
I found this issue while investigating #5942.

## Version
hdf5 git tip as of today (476c60961c4148c31e738cd1e3e9a253c6917cf8)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.