google-deepmind / google-deepmind/torch-hdf5
Cant read lzf compressed files
- Dominant language
- Lua
- Stars
- 241
- Forks
- 125
- PR merge metrics
- No merged PRs in 30d
Description
I have a separate h5py script to generate a datafile and I initially set compression="lzf". The torch-hd5 version failed to read data from that file with the error mentioned below. I changed my script to set compression="gzip" and there were no problems reading it. Please update the docs to specify the compression formats supported. Thanks!
Error when reading lzf compressed file:
```
HDF5-DIAG: Error detected in HDF5 (1.8.5-patch1) thread 0:
#000: ../../src/H5Dio.c line 174 in H5Dread(): can't read data
major: Dataset
minor: Read failed
#001: ../../src/H5Dio.c line 404 in H5D_read(): can't read data
major: Dataset
minor: Read failed
#002: ../../src/H5Dchunk.c line 1724 in H5D_chunk_read(): unable to read raw data chunk
major: Low-level I/O
minor: Read failed
#003: ../../src/H5Dchunk.c line 2737 in H5D_chunk_lock(): data pipeline read failed
major: Data filters
minor: Filter operation failed
#004: ../../src/H5Z.c line 1095 in H5Z_pipeline(): required filter is not registered
major: Data filters
minor: Read failed
```
Contributor guide
Research direction
No documentation file or test is named in the issue. Start by locating the project's documentation describing HDF5 compression support and compare it with the reported lzf failure; done means the supported compression formats are explicitly documented, including the observed gzip behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100