JuliaIO / JuliaIO/ObjectFile.jl

`ObjectFile` not thread-safe

Open
#50 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
17
Forks
12
PR merge metrics
No merged PRs in 30d

Description

In the effort of making BinaryBuilder's Auditor parallel I ran into race conditions during access to object handles:

nested task error: EOFError: read end of file
Stacktrace:
  [1] unsafe_read(s::IOStream, p::Ptr{UInt8}, nb::UInt64)
    @ Base ./iostream.jl:427
  [2] unsafe_read
    @ ./io.jl:724 [inlined]
  [3] unsafe_read(s::IOStream, p::Base.RefValue{ELFSection64{ELFHandle{IOStream}}}, n::Int64)
    @ Base ./io.jl:723
  [4] unsafe_unpack(io::IOStream, #unused#::Type{ELFSection64{ELFHandle{IOStream}}}, target::Base.RefValue{ELFSection64{ELFHandle{IOStream}}}, endianness::Symbol, #unused#::Type{StructIO.Default})
    @ StructIO /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/StructIO/c2JPj/src/StructIO.jl:158
  [5] unpack(io::IOStream, T::Type, endianness::Symbol)
    @ StructIO /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/StructIO/c2JPj/src/StructIO.jl:292
  [6] unpack(oh::ELFHandle{IOStream}, T::Type)
    @ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/ObjectHandle.jl:183
  [7] getindex_ref(collection::ELFSections{ELFHandle{IOStream}}, offset::UInt64, stride::UInt16, T::Type, ref_type::Type{SectionRef}, idx::Int64)
    @ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/utils.jl:175
  [8] getindex(sections::ELFSections{ELFHandle{IOStream}}, idx::Int64)
    @ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:60
  [9] iterate(sections::ELFSections{ELFHandle{IOStream}}, idx::Int64) (repeats 2 times)
    @ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:54
 [10] iterate(::Base.Iterators.Filter{ObjectFile.var"#9#10"{Vector{String}}, ELFSections{ELFHandle{IOStream}}})
    @ Base.Iterators ./iterators.jl:445
 [11] iterate
    @ ./generator.jl:44 [inlined]
 [12] grow_to!
    @ ./array.jl:797 [inlined]
 [13] collect(itr::Base.Generator{Base.Iterators.Filter{ObjectFile.var"#9#10"{Vector{String}}, ELFSections{ELFHandle{IOStream}}}, typeof(identity)})
    @ Base ./array.jl:721
 [14] findall
    @ /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:85 [inlined]
 [15] findfirst
    @ /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:103 [inlined]
 [16] findfirst
    @ /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/Section.jl:94 [inlined]
 [17] RPath(oh::ELFHandle{IOStream})
    @ ObjectFile.ELF /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/ELF/ELFDynamicLink.jl:103
 [18] _broadcast_getindex_evalf
    @ ./broadcast.jl:670 [inlined]
 [19] _broadcast_getindex
    @ ./broadcast.jl:643 [inlined]
 [20] _getindex
    @ ./broadcast.jl:667 [inlined]
 [21] _broadcast_getindex
    @ ./broadcast.jl:642 [inlined]
 [22] getindex
    @ ./broadcast.jl:597 [inlined]
 [23] copy
    @ ./broadcast.jl:899 [inlined]
 [24] materialize
    @ ./broadcast.jl:860 [inlined]
 [25] (::BinaryBuilder.Auditor.var"#10#11")(ohs::Vector{ELFHandle{IOStream}})
    @ BinaryBuilder.Auditor /data/staticfloat/bb_azp/rootfs/agent_2/_work/1/s/src/auditor/dynamic_linkage.jl:80
 [26] readmeta(f::BinaryBuilder.Auditor.var"#10#11", file::String)
    @ ObjectFile /data/staticfloat/bb_azp/rootfs/agent_2/depot/packages/ObjectFile/GXCd7/src/Abstract/ObjectHandle.jl:160
 [27] _canonical_rpaths
    @ /data/staticfloat/bb_azp/rootfs/agent_2/_work/1/s/src/auditor/dynamic_linkage.jl:79 [inlined]
 [28] update_linkage(prefix::Prefix, platform::Platform, path::String, old_libpath::SubString{String}, new_libpath::String; verbose::Bool, subdir::String)
    @ BinaryBuilder.Auditor /data/staticfloat/bb_azp/rootfs/agent_2/_work/1/s/src/auditor/dynamic_linkage.jl:469
 [29] macro expansion
    @ /data/staticfloat/bb_azp/rootfs/agent_2/_work/1/s/src/Auditor.jl:444 [inlined]
 [30] (::BinaryBuilder.Auditor.var"#127#128"{Platform, Bool, Bool, Bool, String, ELFHandle{IOStream}, Prefix, Vector{String}, Dict{SubString{String}, SubString{String}}, String, Base.Threads.Atomic{Bool}, SubString{String}})()
    @ BinaryBuilder.Auditor ./threadingconstructs.jl:178

My understanding is that reading the handle isn't thread-safe. I'm working around this in BB (I think), butit'd be nicer if object files handles were thread-safe by default (e.g. with a lock to automatically acquired when performing unsafe operations).

Edit: however the stacktrace points to this line in a serial function (called inside a threaded region) where however there's a single object handle for the file in question, so I'm not entirely sure where the race condition is coming from 🤔

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Abstract/ObjectHandle.jl, especially the readmeta and unpack paths, then inspect Abstract/Section.jl, utils.jl, and ELF/ELFDynamicLink.jl around the stack trace. Reproduce concurrent reads through the BinaryBuilder Auditor path in dynamic_linkage.jl:80. Done means the thread-safety failure is reproduced and object-handle access behaves correctly under concurrent use.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
compilers, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.