JuliaPy / JuliaPy/PyCall.jl

PyCall + Distributed: cannot pickle 'traceback' object

Aperta
#897 3 commenti 2 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Julia
Stelle
1.5k
Fork
186
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Hi, I posted on [Discourse](https://discourse.julialang.org/t/distributed-pycall-cannot-see-exceptions-cannot-pickle-traceback-object/59802) but no answer so far, so maybe it's better here?
I have trouble getting the error type/message when running PyCall with `@distributed` (when exceptions are raised in Python). It seems that it's the stack trace which poses problem, but I don't know if it is a known limitation or a bug? From PyCall or from Distributed?

```julia
julia> using Distributed

julia> addprocs(1)
1-element Vector{Int64}:
2

julia> @everywhere using PyCall

julia> fetch(@spawnat 2 py"""
raise ZeroDivisionError
""")
┌ Error: Fatal error on process 2
│ exception =
│ PyError ($(Expr(:escape, :(ccall(#= /home/me/.julia/packages/PyCall/BD546/src/pyfncall.jl:43 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw)))))
│ TypeError("cannot pickle 'traceback' object")

│ Stacktrace:
│ [1] pyerr_check
│ @ ~/.julia/packages/PyCall/BD546/src/exception.jl:62 [inlined]
│ [2] pyerr_check
│ @ ~/.julia/packages/PyCall/BD546/src/exception.jl:66 [inlined]
│ [3] _handle_error(msg::String)
│ @ PyCall ~/.julia/packages/PyCall/BD546/src/exception.jl:83
│ [4] macro expansion
│ @ ~/.julia/packages/PyCall/BD546/src/exception.jl:97 [inlined]
│ [5] #107
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:43 [inlined]
│ [6] disable_sigint
│ @ ./c.jl:458 [inlined]
│ [7] __pycall!
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:42 [inlined]
│ [8] _pycall!(ret::PyObject, o::PyObject, args::Tuple{PyObject}, nargs::Int64, kw::Ptr{Nothing})
│ @ PyCall ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:29
│ [9] _pycall!
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:11 [inlined]
│ [10] #pycall#112
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:80 [inlined]
│ [11] pycall
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:80 [inlined]
│ [12] serialize(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, pyo::PyObject)
│ @ PyCall ~/.julia/packages/PyCall/BD546/src/serialize.jl:14
│ [13] serialize_any(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, x::Any)
│ @ Serialization /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Serialization/src/Serialization.jl:649
│ [14] serialize(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, x::Any)
│ @ Serialization /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Serialization/src/Serialization.jl:628
│ [15] serialize(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, ex::CapturedException)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/clusterserialize.jl:205
│ [16] serialize_any(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, x::Any)
│ @ Serialization /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Serialization/src/Serialization.jl:649
│ [17] serialize(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, x::Any)
│ @ Serialization /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Serialization/src/Serialization.jl:628
│ [18] serialize_msg(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, o::Distributed.ResultMsg)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/messages.jl:78
│ [19] #invokelatest#2
│ @ ./essentials.jl:708 [inlined]
│ [20] invokelatest
│ @ ./essentials.jl:706 [inlined]
│ [21] send_msg_(w::Distributed.Worker, header::Distributed.MsgHeader, msg::Distributed.ResultMsg, now::Bool)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/messages.jl:174
│ [22] send_msg_now
│ @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/messages.jl:118 [inlined]
│ [23] send_msg_now(s::Sockets.TCPSocket, header::Distributed.MsgHeader, msg::Distributed.ResultMsg)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/messages.jl:113
│ [24] deliver_result(sock::Sockets.TCPSocket, msg::Symbol, oid::Distributed.RRID, value::RemoteException)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:95
│ [25] macro expansion
│ @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:286 [inlined]
│ [26] (::Distributed.var"#105#107"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()
│ @ Distributed ./task.jl:406
└ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:99
Worker 2 terminated.ERROR:
ProcessExitedException(2)
Stacktrace:
[1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
@ Base ./task.jl:705
[2] wait
@ ./task.jl:764 [inlined]
[3] wait(c::Base.GenericCondition{ReentrantLock})
@ Base ./condition.jl:106
[4] take_buffered(c::Channel{Any})
@ Base ./channels.jl:389
[5] take!(c::Channel{Any})
@ Base ./channels.jl:383
[6] take!(::Distributed.RemoteValue)
@ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:599
[7] #remotecall_fetch#143
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:390 [inlined]
[8] remotecall_fetch(f::Function, w::Distributed.Worker, args::Distributed.RRID)
@ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:386
[9] #remotecall_fetch#146
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:421 [inlined]
[10] remotecall_fetch
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:421 [inlined]
[11] call_on_owner
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:494 [inlined]
[12] fetch(r::Future)
@ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:533
[13] top-level scope
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/macros.jl:99
```

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Reproduce the failure with Distributed, addprocs(1), @everywhere using PyCall, and the @spawnat example. Start by reading PyCall's src/serialize.jl and exception.jl alongside the Distributed serialization frames shown in the report. Done means the remote Python exception's type and message can be obtained without the worker terminating.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
distributed-systems
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.