google-deepmind / google-deepmind/torch-hdf5
HDF5 unexpectedly fails
- Dominant language
- Lua
- Stars
- 241
- Forks
- 125
- PR merge metrics
- No merged PRs in 30d
Description
When I run the following simple code, the hdf5 throws some errors [when i> 5600].
```
require 'torch'
require 'hdf5'
local chunked_h5_file = hdf5.open('test.h5', 'w')
local ii_data = torch.Tensor()
local dtype= 'torch.FloatTensor'
for i = 1, 10000 do
chunked_h5_file:write('vi'..tostring(i),ii_data:resize(10,10):zero():type(dtype))
end
```
Here is the error message:
```
HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 140328551929728:
#000: ../../../src/H5D.c line 170 in H5Dcreate2(): unable to create dataset
major: Dataset
minor: Unable to initialize object
#001: ../../../src/H5Dint.c line 439 in H5D__create_named(): unable to create and link to dataset
major: Dataset
minor: Unable to initialize object
#002: ../../../src/H5L.c line 1638 in H5L_link_object(): unable to create new link to object
major: Links
minor: Unable to initialize object
#003: ../../../src/H5L.c line 1882 in H5L_create_real(): can't insert link
major: Symbol table
minor: Unable to insert object
#004: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#005: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#006: ../../../src/H5L.c line 1685 in H5L_link_cb(): unable to create object
major: Object header
minor: Unable to initialize object
#007: ../../../src/H5O.c line 3015 in H5O_obj_create(): unable to open object
major: Object header
minor: Can't open object
#008: ../../../src/H5Doh.c line 293 in H5O__dset_create(): unable to create dataset
major: Dataset
minor: Unable to initialize object
#009: ../../../src/H5Dint.c line 1048 in H5D__create(): can't update the metadata cache
major: Dataset
minor: Unable to initialize object
#010: ../../../src/H5Dint.c line 802 in H5D__update_oh_info(): unable to create dataset object header
major: Dataset
minor: Unable to initialize object
#011: ../../../src/H5O.c line 1257 in H5O_create(): unable to cache object header
major: Object header
minor: Unable to insert object
#012: ../../../src/H5AC.c line 986 in H5AC_insert_entry(): H5C_insert_entry() failed
major: Object cache
minor: Unable to insert metadata into cache
#013: ../../../src/H5C.c line 2699 in H5C_insert_entry(): H5C_make_space_in_cache failed.
major: Object cache
minor: Unable to insert metadata into cache
#014: ../../../src/H5C.c line 8223 in H5C_make_space_in_cache(): unable to flush entry
major: Object cache
minor: Unable to flush data from cache
#015: ../../../src/H5C.c line 7749 in H5C_flush_single_entry(): unable to flush entry
major: Object cache
minor: Unable to flush data from cache
#016: ../../../src/H5Ocache.c line 487 in H5O_flush(): unable to write object header chunk to disk
major: Object header
minor: Write failed
#017: ../../../src/H5Fio.c line 158 in H5F_block_write(): write through metadata accumulator failed
major: Low-level I/O
minor: Write failed
#018: ../../../src/H5Faccum.c line 640 in H5F_accum_write(): file write failed
major: Low-level I/O
minor: Write failed
#019: ../../../src/H5FDint.c line 185 in H5FD_write(): driver write request failed
major: Virtual File Layer
minor: Write failed
#020: ../../../src/H5FDsec2.c line 822 in H5FD_sec2_write(): file write failed: time = Wed Jul 20 05:29:00 2016
, filename = '/hdfs/test.h5', file descriptor = 3, errno = 5, error message = 'Input/output error', buf = 0x44211a8, total write size = 144, bytes this sub-write = 144, bytes actually written = 18446744073709551615, offset = 274432
major: Low-level I/O
minor: Write failed
HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 140328551929728:
#000: ../../../src/H5Dio.c line 234 in H5Dwrite(): can't prepare for writing data
major: Dataset
minor: Write failed
#001: ../../../src/H5Dio.c line 266 in H5D__pre_write(): not a dataset
major: Invalid arguments to routine
minor: Inappropriate type
../pkg/torch/install/bin/luajit: ../pkg/torch/install/share/lua/5.1/hdf5/group.lua:115: Error writing data vi5686 to [HDF5Group 33554432 /]
stack traceback:
[C]: in function 'error'
.../pkg/torch/install/share/lua/5.1/hdf5/group.lua:115: in function '_writeData'
.../pkg/torch/install/share/lua/5.1/hdf5/group.lua:292: in function '_write_or_append'
.../pkg/torch/install/share/lua/5.1/hdf5/group.lua:255: in function
...g/pkg/torch/install/share/lua/5.1/hdf5/file.lua:69: in function '_write_or_append'
...g/pkg/torch/install/share/lua/5.1/hdf5/file.lua:43: in function 'write'
check_hd5.lua:9: in main chunk
[C]: in function 'dofile'
.../pkg/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x00406670
Segmentation fault (core dumped)
```
I have run this codes on different machines, it fails only on one machine. I installed fresh Ubuntu 14.04. Don't know what is the problem. I tried to update different libraries and packages but no luck.
Any help would be appreciated.
Contributor guide
Assessment
This issue has not been assessed yet.