PyCall crashes remote workers on 0.5
- 主要言語
- Julia
- スター
- 1.5k
- フォーク
- 186
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
Having an issue where calling `@everywhere using PyCall` crashes out remote workers.
I had this issue before:
- deleted the PyCall directory,
- recloned the git repo in my `~/.julia/v0.5` folder
- ran `Pkg.build("PyCall")`
- used `@everywhere using PyCall` and it worked
- exited later due to some other issue after using some @pyimported stuff successfully
- restarted Julia
- ran into the issue again, hence the post here.
Running `using PyCall` on the master process doesn't produce an error.
Here's a dump of what went on:
```
_ _ _(_)_ | A fresh approach to technical computing
(_) | (_) (_) | Documentation: http://docs.julialang.org
_ _ _| |_ __ _ | Type "?help" for help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 0.5.0 (2016-09-19 18:14 UTC)
_/ |\__'_|_|_|\__'_| | Official http://julialang.org/ release
|__/ | x86_64-pc-linux-gnu
julia> # SETUP THE CLUSTER
using ClusterManagers
julia> using ClusterUtils
WARNING: deprecated syntax "[a=>b for (a,b) in c]".
Use "Dict(a=>b for (a,b) in c)" instead.
WARNING: Method definition del_client(Any, Any, Any) in module Base at multi.jl:865 overwritten in module ClusterUtils at /home/mcp50/.julia/v0.5/ClusterUtils/src/ClusterUtils.jl:366.
julia> numnodes = 2
2
julia> nodecore = 1
1
julia> numprocs = numnodes*nodecore
2
julia> remotes = addprocs(SlurmManager(numprocs), partition="mrc-bsu-tesla", account="MRC-BSU-SL2-GPU", t="24:00:00", nodes=numnodes);
srun: job 3711471 queued and waiting for resources
srun: job 3711471 has been allocated resources
connecting to worker 2 out of 2
julia> # ENSURE WE USE MULTITHREADING
if VERSION @everywhere FFTW.set_num_threads(12)
julia> @everywhere using PyCall
Worker 2 terminated.
Worker 3 terminated.ERROR (unhandled task failure): EOFError: read end of file
ERROR: ProcessExitedException()
in yieldto(::Task, ::ANY) at ./event.jl:136
in wait() at ./event.jl:169
in wait(::Condition) at ./event.jl:27
in wait(::Channel{Any}) at ./channels.jl:92
in take!(::Channel{Any}) at ./channels.jl:73
in #remotecall_fetch#606(::Array{Any,1}, ::Function, ::Function, ::Base.Worker) at ./multi.jl:1066
in remotecall_fetch(::Function, ::Base.Worker) at ./multi.jl:1062
in #remotecall_fetch#609(::Array{Any,1}, ::Function, ::Function, ::Int64) at ./multi.jl:1080
in remotecall_fetch(::Function, ::Int64) at ./multi.jl:1080
in (::##14#16)() at ./multi.jl:1959
...and 2 other exceptions.
in sync_end() at ./task.jl:311
in macro expansion; at ./multi.jl:1968 [inlined]
in anonymous at ./:?
ERROR (unhandled task failure): EOFError: read end of file
julia> srun: error: mrc-bsu-tesla2: task 1: Segmentation fault
srun: error: mrc-bsu-tesla1: task 0: Segmentation fault
```
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
調査の方向性
Start by reproducing the reported Julia 0.5 cluster setup with ClusterManagers, ClusterUtils, SlurmManager, and two workers, comparing `using PyCall` on the master with `@everywhere using PyCall`. Use the worker termination and Slurm segmentation-fault output as the initial failure signal; done means remote workers remain available after PyCall is loaded.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python
- 領域
- distributed-systems
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- 説明が足りない
- 初心者へのやさしさ
- 25/100