JuliaParallel / JuliaParallel/MPI.jl

Mysterious serialization crash during bcast

Open
#881 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
439
Forks
127
Avg merge
19h 6m
Merged PRs (30d)
17

Description

Hi there! We have hit this cryptic error in MPI.jl's `bcast(..)` pasted below during a large MPI computation (used by [Pigeons.jl](https://pigeons.run/dev/) running an exo planet posterior inference simulation from @sefffal 's [Octofitter.jl](https://sefffal.github.io/Octofitter.jl/dev/)).

Launching the simulation a second time the error did not pop up, but as we will do more large scale simulations on this model, we are interested in figure out potential causes for this error.

Have you guys seen that before? Could a network glitch cause that? Thank you for your time.

```
ERROR: LoadError: UndefRefError: access to undefined reference
Stacktrace:
[1] getindex
@ ./essentials.jl:13 [inlined]
[2] getindex
@ ./abstractarray.jl:1291 [inlined]
[3] desertag
@ /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:99 [inlined]
[4] handle_deserialize(s::Serializer{IOBuffer}, b::Int32)
@ Serialization /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:852
[5] deserialize
@ /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:814 [inlined]
[6] deserialize_dict(s::Serializer{IOBuffer}, T::Type{Dict{Pair{Int64, Int64}, Any}})
@ Serialization /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:1528
[7] deserialize(s::Serializer{IOBuffer}, T::Type{Dict{Pair{Int64, Int64}, Any}})
@ Serialization /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:1536
[8] handle_deserialize(s::Serializer{IOBuffer}, b::Int32)
@ Serialization /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:883
[9] deserialize(s::Serializer{IOBuffer}, t::DataType)
@ Serialization /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:1513
[10] handle_deserialize(s::Serializer{IOBuffer}, b::Int32)
@ Serialization /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:878
[11] deserialize(s::Serializer{IOBuffer})
@ Serialization /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:814
[12] handle_deserialize(s::Serializer{IOBuffer}, b::Int32)
@ Serialization /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:920
[13] deserialize
@ /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:814 [inlined]
[14] deserialize
@ /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/julia/1.10.0/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:801 [inlined]
[15] deserialize
@ ~/.julia/packages/MPI/is7GN/src/MPI.jl:17 [inlined]
[16] bcast(obj::Nothing, root::Int32, comm::MPI.Comm)
@ MPI ~/.julia/packages/MPI/is7GN/src/collective.jl:101
[17] bcast
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.