Randomly occuring error on reading large MAT files

Open
#81 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
julia
Domain
data

Research direction

Reproduce the failure by repeatedly reading multiple large MAT files with matread, using the reported Julia 0.6.2 and Windows setup if available. Start with MAT_HDF5.jl at m_read and the attribute-reading calls shown in the stack trace, then inspect the HDF5.jl interaction. Done means repeated reads no longer leave subsequent files unreadable and a regression test covers the failure.

Written by the indexing model from the issue text.

Description

There is an error which occurs every time I try to read multiple large MAT files.
Strangely enough, the error occurs only from time to time, and once it appears I can't load any further files without restarting Julia. I am running the newest versions of all packages and the standard build of Julia 0.6.2 on Windows.

In the Atom IDE I see the following:

Error getting attribute type
h5a_get_type at HDF5.jl:2152 [inlined]
datatype(::HDF5.HDF5Attribute) at HDF5.jl:1125
read(::HDF5.HDF5Attribute, ::Type{String}) at HDF5.jl:1323
a_read(::HDF5.HDF5Dataset, ::String) at HDF5.jl:1229
m_read(::HDF5.HDF5Dataset) at MAT_HDF5.jl:153
m_read(::HDF5.HDF5Dataset) at MAT_HDF5.jl:163
m_read(::HDF5.HDF5Group) at MAT_HDF5.jl:239
read(::MAT.MAT_HDF5.MatlabHDF5File, ::String) at MAT_HDF5.jl:258
read(::MAT.MAT_HDF5.MatlabHDF5File) at datafile.jl:45
matread(::String) at MAT.jl:132
load_matlab(::String, ::String) at data_preparation.jl:93
macro expansion at load_matlab_traces_to_jld2.jl:59 [inlined]
anonymous at <missing>:?
include_string(::String, ::String) at loading.jl:522
include_string(::String, ::String, ::Int64) at eval.jl:30
include_string(::Module, ::String, ::String, ::Int64, ::Vararg{Int64,N} where N) at eval.jl:34
(::Atom.##102#107{String,Int64,String})() at eval.jl:82
withpath(::Atom.##102#107{String,Int64,String}, ::String) at utils.jl:30
withpath(::Function, ::String) at eval.jl:38
hideprompt(::Atom.##101#106{String,Int64,String}) at repl.jl:67
macro expansion at eval.jl:80 [inlined]
(::Atom.##100#105{Dict{String,Any}})() at task.jl:80

And the console prints out

HDF5-DIAG: Error detected in HDF5 (1.8.13) thread 0:
  #000: /home/abuild/rpmbuild/BUILD/hdf5-1.8.13/src/H5A.c line 1302 in H5Aget_type(): unable to regi
ster datatype ID
    major: Object atom
    minor: Unable to register new atom
  #001: /home/abuild/rpmbuild/BUILD/hdf5-1.8.13/src/H5I.c line 895 in H5I_register(): can't insert I
D node into skip list
    major: Object atom
    minor: Unable to insert object
  #002: /home/abuild/rpmbuild/BUILD/hdf5-1.8.13/src/H5SL.c line 995 in H5SL_insert(): can't create n
ew skip list node
    major: Skip Lists
    minor: Unable to insert object
  #003: /home/abuild/rpmbuild/BUILD/hdf5-1.8.13/src/H5SL.c line 687 in H5SL_insert_common(): can't i
nsert duplicate key
    major: Skip Lists
    minor: Unable to insert object
Dominant language
Julia
Stars
293
Forks
79
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from JuliaIO/MAT.jl

All issues in JuliaIO/MAT.jl

Similar issues

More Julia issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.