JuliaParallel / JuliaParallel/DistributedArrays.jl
DistributedArrays is not thread-safe
- Lenguaje dominante
- Julia
- Estrellas
- 205
- Forks
- 34
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
Due to the use of globals such as `DistributedArrays.registry`, currently DistributedArrays is not thread-safe:
```julia
julia> versioninfo()
Julia Version 1.11.6
Commit 9615af0f269 (2025-07-09 12:58 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: macOS (arm64-apple-darwin24.0.0)
CPU: 10 × Apple M2 Pro
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, apple-m2)
Threads: 6 default, 0 interactive, 3 GC (on 6 virtual cores)
Environment:
JULIA_PKG_USE_CLI_GIT = true
JULIA_PKG_PRESERVE_TIERED_INSTALLED = true
julia> using DistributedArrays
julia> v = [rand(10) for _ in 1:1000];
julia> @sync for vi in v
Threads.@spawn distribute(vi)
end;
ERROR: TaskFailedException
nested task error: TaskFailedException
nested task error: AssertionError: Multiple concurrent writes to Dict detected!
Stacktrace:
[1] _setindex!
@ ./dict.jl:337 [inlined]
[2] setindex!(h::Dict{Any, Nothing}, v0::Nothing, key::Tuple{Int64, Int64})
@ Base ./dict.jl:363
[3] push!
@ ./set.jl:137 [inlined]
[4] DArray{…}(id::Tuple{…}, dims::Tuple{…}, pids::Vector{…}, indices::Vector{…}, cuts::Vector{…}, lp::Vector{…})
@ DistributedArrays ~/.julia/packages/DistributedArrays/SxLCk/src/darray.jl:47
[5] construct_localparts(init::DistributedArrays.var"#73#75"{…}, id::Tuple{…}, dims::Tuple{…}, pids::Vector{…}, idxs::Vector{…}, cuts::Vector{…}; T::Nothing, A::Nothing)
@ DistributedArrays ~/.julia/packages/DistributedArrays/SxLCk/src/darray.jl:126
[6] construct_localparts(init::Function, id::Tuple{…}, dims::Tuple{…}, pids::Vector{…}, idxs::Vector{…}, cuts::Vector{…})
@ DistributedArrays ~/.julia/packages/DistributedArrays/SxLCk/src/darray.jl:117
[7] #invokelatest#2
@ ./essentials.jl:1055 [inlined]
[8] invokelatest
@ ./essentials.jl:1052 [inlined]
[9] #153
@ ~/.julia/juliaup/julia-1.11.6+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:425 [inlined]
[10] run_work_thunk(thunk::Distributed.var"#153#154"{…}, print_error::Bool)
@ Distributed ~/.julia/juliaup/julia-1.11.6+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Distributed/src/process_messages.jl:70
[11] #remotecall_fetch#158
@ ~/.julia/juliaup/julia-1.11.6+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:450 [inlined]
[12] remotecall_fetch
@ ~/.julia/juliaup/julia-1.11.6+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:449 [inlined]
[13] remotecall_fetch
@ ~/.julia/juliaup/julia-1.11.6+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:492 [inlined]
[14] (::DistributedArrays.var"#1#3"{…})()
@ DistributedArrays ~/.julia/packages/DistributedArrays/SxLCk/src/darray.jl:88
Stacktrace:
[1] #remotecall_fetch#158
@ ~/.julia/juliaup/julia-1.11.6+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:451 [inlined]
[2] remotecall_fetch
@ ~/.julia/juliaup/julia-1.11.6+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:449 [inlined]
[3] remotecall_fetch
@ ~/.julia/juliaup/julia-1.11.6+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:492 [inlined]
[4] (::DistributedArrays.var"#1#3"{…})()
@ DistributedArrays ~/.julia/packages/DistributedArrays/SxLCk/src/darray.jl:88
Stacktrace:
[1] sync_end(c::Channel{Any})
@ Base ./task.jl:466
[2] macro expansion
@ ./task.jl:499 [inlined]
[3] DArray(id::Tuple{…}, init::Function, dims::Tuple{…}, pids::Vector{…}, idxs::Vector{…}, cuts::Vector{…})
@ DistributedArrays ~/.julia/packages/DistributedArrays/SxLCk/src/darray.jl:83
[4] DArray(init::Function, dims::Tuple{Int64}, procs::Vector{Int64}, dist::Vector{Int64})
@ DistributedArrays ~/.julia/packages/DistributedArrays/SxLCk/src/darray.jl:177
[5] distribute(A::Vector{Float64}; procs::Vector{Int64}, dist::Vector{Int64})
@ DistributedArrays ~/.julia/packages/DistributedArrays/SxLCk/src/darray.jl:550
[6] distribute(A::Vector{Float64})
@ DistributedArrays ~/.julia/packages/DistributedArrays/SxLCk/src/darray.jl:540
[7] (::var"#7#8"{Vector{Float64}})()
@ Main ./REPL[6]:2
...and 3 more exceptions.
Stacktrace:
[1] sync_end(c::Channel{Any})
@ Base ./task.jl:466
[2] macro expansion
@ task.jl:499 [inlined]
[3] top-level scope
@ REPL[6]:1
Some type information was truncated. Use `show(err)` to see complete types.
```
AFAICT this could be fixed by using task-local states (`OncePerTask` on Julia >= 1.12).
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Línea de trabajo
Start with src/darray.jl, especially the DArray constructor and construct_localparts paths shown in the stack trace, and inspect uses of DistributedArrays.registry. Run the concurrent distribute reproducer from the issue first. Done means concurrent calls no longer trigger the Dict concurrent-write assertion and the relevant behavior is covered by tests.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- julia
- Área
- data, distributed-systems
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 35/100