h5repack issues including from static install

(Created from a helpdesk issue)

I have been trying to use h5repack in HDF5 1.14.0 and 1.14.5 and in particular when we have a static-only build of libhdf5.a and associated binary tools.

I had the impression that a compression plugin should be able to work with a static-only install of HDF5 and, in particular, h5repack should be able to use the plugin when requested via the -f argument. However, that is turning out not to be my experience.

gcc -I. -I/usr/gapps/silo/zfp/1.0.1/blueos_3_ppc64le_ib_p9-gcc-8.3.1/include -I/usr/gapps/silo/hdf5/1.14.5/blueos_3_ppc64le_ib_p9_gcc.8.3.1/include -shared -fPIC -o plugin/libh5zzfp.so H5Zzfp.c -L/usr/gapps/silo/zfp/1.0.1/blueos_3_ppc64le_ib_p9-gcc-8.3.1/lib -lzfp -lsz -lm -lz -ldl -Wl,-Bsymbolic -Wl,-undefined,dynamic_lookup

I will identify a few issues I am seeing in bullets below…

1. About h5repack behavior when it cannot load a mandatory filter:
* Specifying -f argument with mandatory filter flag AND naming a specific dataset to apply the filter to DOES NOT issue any error message or error code when it fails. Specfically, I tried…

env HDF5_PLUGIN_PATH=`pwd`/foo /g/g11/miller86/tmp/build-hdf5-1.14.5-blueos_3_ppc64le_ib_p9-gcc-8.3.1/my_install/bin/h5repack -f .silo/#000015:UD=32013,0,4,1,0,0,1074528256 /usr/gapps/visit/data/wave0000.silo foo.silo

* note that `pwd`/foo is an invalid path. So, it can find no plugin with id 32013. the command, I specifically request dataset .silo/#000015 be compressed with that filter and I give the optional flag a 0 value…meaning mandatory. I get no error message or error code from h5repack
* It should probably provide an error message that it could not load the requested filter since the filter is mandatory in this case.
* I get that it may be problematic for h5repack to error if -f is specified but without targeting specific objects because then it is up to the filter’s can_apply method to decide which objects it can be applied to. OTOH, what does a 0 optional flag value mean in that case? It could potentially mean to list all the objects the filter could NOT be applied to upon completion but WITHOUT generating any error return value from h5repack. Where as, specifying a 1 optional flag means you don’t care to get that list. That’s just an idea on my part and not an actual request. So, take it or leave it.
* This likely has nothing to do with static-only or shared/static builds.
1. About static-only h5repack behavior with either mandatory or optional filter:
* This is very strange but I’ve attempted to compress a specific dataset with the filter both as mandatory and as optional. And, I get behavior that represents kinda sorta the reverse of what I would expect. And, it appears to be in a “limbo” state…h5ls reports that the dataset has been filtered. But, it doesn’t show the filter by name, only by id and the dataset is the same size as uncompressed. I show 2 runs below. The first uses 0 for optional flag…meaning mandatory. The second uses 1 for optional flag meaning optional. Note however that h5ls on that dataset for the first shows that the dataset was never filtered whereas for the second case, it reports it was filtered but then it doesn’t report it the way I would normally see it if it had truly loaded the plugin correct. I think it failed to load the plugin.

lassen708{miller86}835: env HDF5_PLUGIN_PATH=`pwd`/plugin /g/g11/miller86/tmp/build-hdf5-1.14.5-blueos_3_ppc64le_ib_p9-gcc-8.3.1/my_install/bin/h5repack -f .silo/#000015:UD=32013,0,4,1,0,0,1074528256 /usr/gapps/visit/data/wave0000.silo foo.silo

lassen708{miller86}836: h5ls -vlr foo.silo/.silo/#000015 Opened "foo.silo" with sec2 driver.

.silo/#000015 Dataset {101/101, 11/11, 16/16}

Location: 1:491964

Links: 1

Storage: 71104 logical bytes, 71104 allocated bytes, 100.00% utilization

Type: native float

lassen708{miller86}837: env HDF5_PLUGIN_PATH=`pwd`/plugin /g/g11/miller86/tmp/build-hdf5-1.14.5-blueos_3_ppc64le_ib_p9-gcc-8.3.1/my_install/bin/h5repack -f .silo/#000015:UD=32013,1,4,1,0,0,1074528256 /usr/gapps/visit/data/wave0000.silo foo.silo

lassen708{miller86}838: h5ls -vlr foo.silo/.silo/#000015 Opened "foo.silo" with sec2 driver.

.silo/#000015 Dataset {101/101, 11/11, 16/16}

Location: 1:491964

Links: 1

Chunks: {101, 11, 16} 71104 bytes

Storage: 71104 logical bytes, 71104 allocated bytes, 100.00% utilization

Filter-0: method-32013 OPT {1, 0, 0, 1074528256}

Type: native float

1. About static-only h5repack behavior: I am not able to get this to work. I have added an `abort()` call in my compression plugin as part of the `can_apply()` method and it never aborts. So, I believe it is never loading the plugin.