JuliaParallel / JuliaParallel/MPI.jl
test_spawn.jl fails (tested on Cray system with Cray mpt 7.7.10)
- Dominant language
- Julia
- Stars
- 439
- Forks
- 127
- Avg merge
- 19h 6m
- Merged PRs (30d)
- 17
Description
That's maybe because as one cannot spawn whatever you want when allocated with SLURM.
```
srun: Warning: can't run 1 processes on 4 nodes, setting nnodes to 1
Rank 0 [Fri May 8 18:08:48 2020] [c0-0c0s0n0] Fatal error in MPI_Comm_spawn: Other MPI error, error stack:
MPI_Comm_spawn(144).........: MPI_Comm_spawn(cmd="/scratch/snx3000tds/omlins/9_soft_install_julia_1_4/software/Julia/1.4.1-CrayGNU-19.10-cuda-10.1/bin/julia", argv=0x2aaabada7290, maxprocs=3, MPI_INFO_NULL, root=0, MPI_COMM_WORLD, intercomm=0x2aaabb379050, errors=0x2aaaba2c2f90) faile
MPID_Comm_spawn_multiple(70): Function MPID_Comm_spawn_multiple not implemented
signal (6): Aborted
in expression starting at /users/omlins/.julia/1.4.1/dom-gpu/dev/MPI/test/test_spawn.jl:11
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
MPID_Abort at /opt/cray/pe/mpt/7.7.10/gni/mpich-gnu/8.2/lib/libmpich.so (unknown line)
MPIR_Handle_fatal_error at /opt/cray/pe/mpt/7.7.10/gni/mpich-gnu/8.2/lib/libmpich.so (unknown line)
MPIR_Err_return_comm at /opt/cray/pe/mpt/7.7.10/gni/mpich-gnu/8.2/lib/libmpich.so (unknown line)
PMPI_Comm_spawn at /opt/cray/pe/mpt/7.7.10/gni/mpich-gnu/8.2/lib/libmpich.so (unknown line)
#Comm_spawn#25 at /users/omlins/.julia/1.4.1/dom-gpu/dev/MPI/src/comm.jl:141
Comm_spawn at /users/omlins/.julia/1.4.1/dom-gpu/dev/MPI/src/comm.jl:137
unknown function (ip: 0x2aaabf21af16)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2158 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1700 [inlined]
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:369
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:458
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:409 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:817
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:911
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:814
jl_parse_eval_all at /buildworker/worker/package_linux64/build/src/ast.c:872
jl_load at /buildworker/worker/package_linux64/build/src/toplevel.c:872
include at ./Base.jl:377
exec_options at ./client.jl:288
_start at ./client.jl:484
jfptr__start_2075.clone_1 at /scratch/snx3000tds/omlins/9_soft_install_julia_1_4/software/Julia/1.4.1-CrayGNU-19.10-cuda-10.1/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux64/build/ui/../src/julia.h:1700 [inlined]
true_main at /buildworker/worker/package_linux64/build/ui/repl.c:96
main at /buildworker/worker/package_linux64/build/ui/repl.c:217
__libc_start_main at /lib64/libc.so.6 (unknown line)
_start at /scratch/snx3000tds/omlins/9_soft_install_julia_1_4/software/Julia/1.4.1-CrayGNU-19.10-cuda-10.1/bin/julia (unknown line)
Allocations: 1701693 (Pool: 1701268; Big: 425); GC: 2
srun: error: nid00000: task 0: Aborted (core dumped)
srun: Terminating job step 1015464.25
ERROR: LoadError: failed process: Process(`srun -C gpu -n 1 /scratch/snx3000tds/omlins/9_soft_install_julia_1_4/software/Julia/1.4.1-CrayGNU-19.10-cuda-10.1/bin/julia --code-coverage=none /users/omlins/.julia/1.4.1/dom-gpu/dev/MPI/test/test_spawn.jl`, ProcessExited(134)) [134]
Stacktrace:
[1] pipeline_error at ./process.jl:525 [inlined]
[2] run(::Cmd; wait::Bool) at ./process.jl:440
[3] run at ./process.jl:438 [inlined]
[4] (::var"#3#7"{Int64,String,String,String,String})(::Cmd) at /users/omlins/.julia/1.4.1/dom-gpu/dev/MPI/test/runtests.jl:37
[5] mpiexec(::var"#3#7"{Int64,String,String,String,String}) at /users/omlins/.julia/1.4.1/dom-gpu/dev/MPI/deps/deps.jl:11
[6] runtests() at /users/omlins/.julia/1.4.1/dom-gpu/dev/MPI/test/runtests.jl:34
[7] top-level scope at /users/omlins/.julia/1.4.1/dom-gpu/dev/MPI/test/runtests.jl:53
[8] include(::Module, ::String) at ./Base.jl:377
[9] exec_options(::Base.JLOptions) at ./client.jl:288
[10] _start() at ./client.jl:484
in expression starting at /users/omlins/.julia/1.4.1/dom-gpu/dev/MPI/test/runtests.jl:53
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.