google-deepmind / google-deepmind/torch-hdf5
Reading data of class STRING(50331965) is unsupported stack traceback
- Dominant language
- Lua
- Stars
- 241
- Forks
- 125
- PR merge metrics
- No merged PRs in 30d
Description
I created my hdf5 file by python with
**embed_info.create_dataset('vocab',(count,),dtype='S10',data=vocab)**
When I open that file using torch, it goes wrong.
Here is the error log:
**torch/install/share/lua/5.1/hdf5/ffi.lua:335: Reading data of class STRING(50331965) is unsupported
stack traceback:
[C]: in function 'error'
/home/lyb/torch/install/share/lua/5.1/hdf5/ffi.lua:335: in function '_getTorchType'
/home/lyb/torch/install/share/lua/5.1/hdf5/dataset.lua:88: in function 'getTensorFactory'
/home/lyb/torch/install/share/lua/5.1/hdf5/dataset.lua:138: in function 'partial'
[string "a= file:read('/embed_info/vocab'):partial({1,2})"]:1: in main chunk
[C]: in function 'xpcall'
/home/lyb/torch/install/share/lua/5.1/trepl/init.lua:669: in function 'repl'
.../lyb/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:199: in main chunk
[C]: at 0x00406670**
Anyone has met such a question?
Contributor guide
Research direction
Reproduce the failure with the Python-created `embed_info/vocab` dataset, then inspect `torch/install/share/lua/5.1/hdf5/ffi.lua` at line 335 and `dataset.lua` at lines 88 and 138 to trace how the STRING type is handled. Done means the reported dataset can be read through the Torch HDF5 interface, or the unsupported case is clearly characterized.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100