JuliaPy / JuliaPy/PyCall.jl

PyCall crashes remote workers on 0.5

Offen
#326 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Julia
Sterne
1.5k
Forks
186
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Having an issue where calling `@everywhere using PyCall` crashes out remote workers.

I had this issue before:
- deleted the PyCall directory,
- recloned the git repo in my `~/.julia/v0.5` folder
- ran `Pkg.build("PyCall")`
- used `@everywhere using PyCall` and it worked
- exited later due to some other issue after using some @pyimported stuff successfully
- restarted Julia
- ran into the issue again, hence the post here.

Running `using PyCall` on the master process doesn't produce an error.

Here's a dump of what went on:

```
_ _ _(_)_ | A fresh approach to technical computing
(_) | (_) (_) | Documentation: http://docs.julialang.org
_ _ _| |_ __ _ | Type "?help" for help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 0.5.0 (2016-09-19 18:14 UTC)
_/ |\__'_|_|_|\__'_| | Official http://julialang.org/ release
|__/ | x86_64-pc-linux-gnu

julia> # SETUP THE CLUSTER
using ClusterManagers

julia> using ClusterUtils

WARNING: deprecated syntax "[a=>b for (a,b) in c]".
Use "Dict(a=>b for (a,b) in c)" instead.
WARNING: Method definition del_client(Any, Any, Any) in module Base at multi.jl:865 overwritten in module ClusterUtils at /home/mcp50/.julia/v0.5/ClusterUtils/src/ClusterUtils.jl:366.

julia> numnodes = 2
2

julia> nodecore = 1
1

julia> numprocs = numnodes*nodecore
2

julia> remotes = addprocs(SlurmManager(numprocs), partition="mrc-bsu-tesla", account="MRC-BSU-SL2-GPU", t="24:00:00", nodes=numnodes);
srun: job 3711471 queued and waiting for resources
srun: job 3711471 has been allocated resources
connecting to worker 2 out of 2
julia> # ENSURE WE USE MULTITHREADING
if VERSION @everywhere FFTW.set_num_threads(12)

julia> @everywhere using PyCall
Worker 2 terminated.
Worker 3 terminated.ERROR (unhandled task failure): EOFError: read end of file
ERROR: ProcessExitedException()
in yieldto(::Task, ::ANY) at ./event.jl:136
in wait() at ./event.jl:169
in wait(::Condition) at ./event.jl:27
in wait(::Channel{Any}) at ./channels.jl:92
in take!(::Channel{Any}) at ./channels.jl:73
in #remotecall_fetch#606(::Array{Any,1}, ::Function, ::Function, ::Base.Worker) at ./multi.jl:1066
in remotecall_fetch(::Function, ::Base.Worker) at ./multi.jl:1062
in #remotecall_fetch#609(::Array{Any,1}, ::Function, ::Function, ::Int64) at ./multi.jl:1080
in remotecall_fetch(::Function, ::Int64) at ./multi.jl:1080
in (::##14#16)() at ./multi.jl:1959

...and 2 other exceptions.

in sync_end() at ./task.jl:311
in macro expansion; at ./multi.jl:1968 [inlined]
in anonymous at ./:?

ERROR (unhandled task failure): EOFError: read end of file
julia> srun: error: mrc-bsu-tesla2: task 1: Segmentation fault
srun: error: mrc-bsu-tesla1: task 0: Segmentation fault
```

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.