JuliaIO / JuliaIO/ObjectFile.jl
`ObjectFile` not thread-safe
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 17
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
In the effort of making BinaryBuilder's Auditor parallel I ran into race conditions during access to object handles:
nested task error: EOFError: read end of file
Stacktrace:
[1] unsafe_read(s::IOStream, p::Ptr{UInt8}, nb::UInt64)
@ Base ./iostream.jl:427
[2] unsafe_read
@ ./io.jl:724 [inlined]
[3] unsafe_read(s::IOStream, p::Base.RefValue{ELFSection64{ELFHandle{IOStream}}}, n::Int64)
@ Base ./io.jl:723
[4] unsafe_unpack(io::IOStream, #unused#::Type{ELFSection64{ELFHandle{IOStream}}}, target::Base.RefValue{ELFSection64{ELFHandle{IOStream}}}, endianness::Symbol, #unused#::Type{StructIO.Default})
@ StructIO /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/StructIO/c2JPj/src/StructIO.jl:158
[5] unpack(io::IOStream, T::Type, endianness::Symbol)
@ StructIO /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/StructIO/c2JPj/src/StructIO.jl:292
[6] unpack(oh::ELFHandle{IOStream}, T::Type)
@ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/ObjectHandle.jl:183
[7] getindex_ref(collection::ELFSections{ELFHandle{IOStream}}, offset::UInt64, stride::UInt16, T::Type, ref_type::Type{SectionRef}, idx::Int64)
@ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/utils.jl:175
[8] getindex(sections::ELFSections{ELFHandle{IOStream}}, idx::Int64)
@ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:60
[9] iterate(sections::ELFSections{ELFHandle{IOStream}}, idx::Int64) (repeats 2 times)
@ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:54
[10] iterate(::Base.Iterators.Filter{ObjectFile.var"#9#10"{Vector{String}}, ELFSections{ELFHandle{IOStream}}})
@ Base.Iterators ./iterators.jl:445
[11] iterate
@ ./generator.jl:44 [inlined]
[12] grow_to!
@ ./array.jl:797 [inlined]
[13] collect(itr::Base.Generator{Base.Iterators.Filter{ObjectFile.var"#9#10"{Vector{String}}, ELFSections{ELFHandle{IOStream}}}, typeof(identity)})
@ Base ./array.jl:721
[14] findall
@ /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:85 [inlined]
[15] findfirst
@ /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:103 [inlined]
[16] findfirst
@ /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:94 [inlined]
[17] RPath(oh::ELFHandle{IOStream})
@ ObjectFile.ELF /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/ELF/ELFDynamicLink.jl:103
[18] _broadcast_getindex_evalf
@ ./broadcast.jl:670 [inlined]
[19] _broadcast_getindex
@ ./broadcast.jl:643 [inlined]
[20] _getindex
@ ./broadcast.jl:667 [inlined]
[21] _broadcast_getindex
@ ./broadcast.jl:642 [inlined]
[22] getindex
@ ./broadcast.jl:597 [inlined]
[23] copy
@ ./broadcast.jl:899 [inlined]
[24] materialize
@ ./broadcast.jl:860 [inlined]
[25] (::BinaryBuilder.Auditor.var"#10#11")(ohs::Vector{ELFHandle{IOStream}})
@ BinaryBuilder.Auditor /data/staticfloat/bb_azp/rootfs/agent_2/_work/1/s/src/auditor/dynamic_linkage.jl:80
[26] readmeta(f::BinaryBuilder.Auditor.var"#10#11", file::String)
@ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/ObjectHandle.jl:160
[27] _canonical_rpaths
@ /data/staticfloat/bb_azp/rootfs/agent_2/_work/1/s/src/auditor/dynamic_linkage.jl:79 [inlined]
[28] update_linkage(prefix::Prefix, platform::Platform, path::String, old_libpath::SubString{String}, new_libpath::String; verbose::Bool, subdir::String)
@ BinaryBuilder.Auditor /data/staticfloat/bb_azp/rootfs/agent_2/_work/1/s/src/auditor/dynamic_linkage.jl:469
[29] macro expansion
@ /data/staticfloat/bb_azp/rootfs/agent_2/_work/1/s/src/Auditor.jl:444 [inlined]
[30] (::BinaryBuilder.Auditor.var"#127#128"{Platform, Bool, Bool, Bool, String, ELFHandle{IOStream}, Prefix, Vector{String}, Dict{SubString{String}, SubString{String}}, String, Base.Threads.Atomic{Bool}, SubString{String}})()
@ BinaryBuilder.Auditor ./threadingconstructs.jl:178
My understanding is that reading the handle isn't thread-safe. I'm working around this in BB (I think), butit'd be nicer if object files handles were thread-safe by default (e.g. with a lock to automatically acquired when performing unsafe operations).
Edit: however the stacktrace points to this line in a serial function (called inside a threaded region) where however there's a single object handle for the file in question, so I'm not entirely sure where the race condition is coming from 🤔
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Abstract/ObjectHandle.jl, especially the readmeta and unpack paths, then inspect Abstract/Section.jl, utils.jl, and ELF/ELFDynamicLink.jl around the stack trace. Reproduce concurrent reads through the BinaryBuilder Auditor path in dynamic_linkage.jl:80. Done means the thread-safety failure is reproduced and object-handle access behaves correctly under concurrent use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- compilers, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100