google-deepmind / google-deepmind/torch-hdf5

Cant read lzf compressed files

Open
#74 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Lua
Stars
241
Forks
125
PR merge metrics
No merged PRs in 30d

Description

I have a separate h5py script to generate a datafile and I initially set compression="lzf". The torch-hd5 version failed to read data from that file with the error mentioned below. I changed my script to set compression="gzip" and there were no problems reading it. Please update the docs to specify the compression formats supported. Thanks!

Error when reading lzf compressed file:

```
HDF5-DIAG: Error detected in HDF5 (1.8.5-patch1) thread 0:
#000: ../../src/H5Dio.c line 174 in H5Dread(): can't read data
major: Dataset
minor: Read failed
#001: ../../src/H5Dio.c line 404 in H5D_read(): can't read data
major: Dataset
minor: Read failed
#002: ../../src/H5Dchunk.c line 1724 in H5D_chunk_read(): unable to read raw data chunk
major: Low-level I/O
minor: Read failed
#003: ../../src/H5Dchunk.c line 2737 in H5D_chunk_lock(): data pipeline read failed
major: Data filters
minor: Filter operation failed
#004: ../../src/H5Z.c line 1095 in H5Z_pipeline(): required filter is not registered
major: Data filters
minor: Read failed
```

Contributor guide

Open the contributing guide

Research direction

No documentation file or test is named in the issue. Start by locating the project's documentation describing HDF5 compression support and compare it with the reported lzf failure; done means the supported compression formats are explicitly documented, including the observed gzip behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.