google-deepmind / google-deepmind/torch-hdf5
Unable to read the "master" hdf5 file after linking smaller hdf5 files to it
- Dominant language
- Lua
- Stars
- 241
- Forks
- 125
- PR merge metrics
- No merged PRs in 30d
Description
I have a couple of small (~20-200GBs) hdf5 files that I link together to get a single hdf5 file (as described [here](https://stackoverflow.com/a/18527687/2838606)). However, I cannot read this larger, master, hdf5 file anymore using torch-hdf5 package when using multiple threads but I can read the smaller files. I wonder if the hdf5 package has issues reading "linked" hdf5 files?
Contributor guide
Research direction
The issue names no repository files or tests. Start by reproducing the linked HDF5 setup with torch-hdf5 under multiple threads, then compare reads from the linked master file with the smaller files; done means identifying whether the failure is specific to linked files and documenting a reproducible result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100