PyCall + Distributed: cannot pickle 'traceback' object
- Langage dominant
- Julia
- Étoiles
- 1.5k
- Forks
- 186
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
Hi, I posted on [Discourse](https://discourse.julialang.org/t/distributed-pycall-cannot-see-exceptions-cannot-pickle-traceback-object/59802) but no answer so far, so maybe it's better here?
I have trouble getting the error type/message when running PyCall with `@distributed` (when exceptions are raised in Python). It seems that it's the stack trace which poses problem, but I don't know if it is a known limitation or a bug? From PyCall or from Distributed?
```julia
julia> using Distributed
julia> addprocs(1)
1-element Vector{Int64}:
2
julia> @everywhere using PyCall
julia> fetch(@spawnat 2 py"""
raise ZeroDivisionError
""")
┌ Error: Fatal error on process 2
│ exception =
│ PyError ($(Expr(:escape, :(ccall(#= /home/me/.julia/packages/PyCall/BD546/src/pyfncall.jl:43 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw)))))
│ TypeError("cannot pickle 'traceback' object")
│
│ Stacktrace:
│ [1] pyerr_check
│ @ ~/.julia/packages/PyCall/BD546/src/exception.jl:62 [inlined]
│ [2] pyerr_check
│ @ ~/.julia/packages/PyCall/BD546/src/exception.jl:66 [inlined]
│ [3] _handle_error(msg::String)
│ @ PyCall ~/.julia/packages/PyCall/BD546/src/exception.jl:83
│ [4] macro expansion
│ @ ~/.julia/packages/PyCall/BD546/src/exception.jl:97 [inlined]
│ [5] #107
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:43 [inlined]
│ [6] disable_sigint
│ @ ./c.jl:458 [inlined]
│ [7] __pycall!
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:42 [inlined]
│ [8] _pycall!(ret::PyObject, o::PyObject, args::Tuple{PyObject}, nargs::Int64, kw::Ptr{Nothing})
│ @ PyCall ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:29
│ [9] _pycall!
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:11 [inlined]
│ [10] #pycall#112
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:80 [inlined]
│ [11] pycall
│ @ ~/.julia/packages/PyCall/BD546/src/pyfncall.jl:80 [inlined]
│ [12] serialize(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, pyo::PyObject)
│ @ PyCall ~/.julia/packages/PyCall/BD546/src/serialize.jl:14
│ [13] serialize_any(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, x::Any)
│ @ Serialization /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Serialization/src/Serialization.jl:649
│ [14] serialize(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, x::Any)
│ @ Serialization /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Serialization/src/Serialization.jl:628
│ [15] serialize(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, ex::CapturedException)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/clusterserialize.jl:205
│ [16] serialize_any(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, x::Any)
│ @ Serialization /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Serialization/src/Serialization.jl:649
│ [17] serialize(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, x::Any)
│ @ Serialization /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Serialization/src/Serialization.jl:628
│ [18] serialize_msg(s::Distributed.ClusterSerializer{Sockets.TCPSocket}, o::Distributed.ResultMsg)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/messages.jl:78
│ [19] #invokelatest#2
│ @ ./essentials.jl:708 [inlined]
│ [20] invokelatest
│ @ ./essentials.jl:706 [inlined]
│ [21] send_msg_(w::Distributed.Worker, header::Distributed.MsgHeader, msg::Distributed.ResultMsg, now::Bool)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/messages.jl:174
│ [22] send_msg_now
│ @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/messages.jl:118 [inlined]
│ [23] send_msg_now(s::Sockets.TCPSocket, header::Distributed.MsgHeader, msg::Distributed.ResultMsg)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/messages.jl:113
│ [24] deliver_result(sock::Sockets.TCPSocket, msg::Symbol, oid::Distributed.RRID, value::RemoteException)
│ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:95
│ [25] macro expansion
│ @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:286 [inlined]
│ [26] (::Distributed.var"#105#107"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()
│ @ Distributed ./task.jl:406
└ @ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:99
Worker 2 terminated.ERROR:
ProcessExitedException(2)
Stacktrace:
[1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
@ Base ./task.jl:705
[2] wait
@ ./task.jl:764 [inlined]
[3] wait(c::Base.GenericCondition{ReentrantLock})
@ Base ./condition.jl:106
[4] take_buffered(c::Channel{Any})
@ Base ./channels.jl:389
[5] take!(c::Channel{Any})
@ Base ./channels.jl:383
[6] take!(::Distributed.RemoteValue)
@ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:599
[7] #remotecall_fetch#143
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:390 [inlined]
[8] remotecall_fetch(f::Function, w::Distributed.Worker, args::Distributed.RRID)
@ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:386
[9] #remotecall_fetch#146
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:421 [inlined]
[10] remotecall_fetch
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:421 [inlined]
[11] call_on_owner
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:494 [inlined]
[12] fetch(r::Future)
@ Distributed /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/remotecall.jl:533
[13] top-level scope
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/macros.jl:99
```
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Piste de recherche
Reproduce the failure with Distributed, addprocs(1), @everywhere using PyCall, and the @spawnat example. Start by reading PyCall's src/serialize.jl and exception.jl alongside the Distributed serialization frames shown in the report. Done means the remote Python exception's type and message can be obtained without the worker terminating.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- python
- Domaine
- distributed-systems
- Type d'issue
- Bug
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- À l'abandon
- Clarté
- À clarifier
- Accessibilité débutants
- 35/100