Can't open gcsfuse-mounted HDF5 file with h5py
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 398
- Forks
- 181
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
First I ran:
gcsfuse <bucket> /tmp/<bucket>
Then, attempting to open an HDF5 file with h5py:
>>> from h5py import *
>>> input = '/tmp/<bucket>/file'
>>> f = File(input, 'r')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "…/venv/lib/python3.6/site-packages/h5py/_hl/files.py", line 312, in __init__
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "…/venv/lib/python3.6/site-packages/h5py/_hl/files.py", line 142, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 78, in h5py.h5f.open
OSError: Unable to open file (file signature not found)
I'm not sure whether this is likely a bug in h5py or gcsfuse; I can see the file with e.g. ls or or.path.getsize, so gcsfuse seems to be doing its job, but my guess is that there is some syscall or access pattern in h5py that in not playing nicely with gcsfuse.
(mostly-irrelevant discussion of this error message in h5py)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the gcsfuse mount and h5py open sequence, starting with h5py/_hl/files.py and the h5py.h5f.open call shown in the traceback. Compare access to the same HDF5 file locally and determine whether the failure belongs to h5py or the mounted Google Cloud Storage filesystem; done should be a confirmed cause with an actionable fix or a clear upstream report.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100