JuliaLang / JuliaLang/Distributed.jl
[Distributed.jl] inconsistent serialization of closures over global vars
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 55
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Description
here's my MWE (tested on 1.9.2):
using Distributed, Serialization
y = 3
f = x -> x + y
worker = only(addprocs(1))
@everywhere worker using Serialization
fs = let
io = IOBuffer()
serialize(io, f)
take!(io)
end;
# error: UndefVarError: `y` not defined
remotecall_fetch(worker, fs, 2) do fs, x
f = deserialize(IOBuffer(fs))
invokelatest(f, x)
end
# succeeds
remotecall_fetch(f, worker, 2)
# now succeeds
remotecall_fetch(worker, fs, 2) do fs, x
f = deserialize(IOBuffer(fs))
invokelatest(f, x)
end
I understand why the first invocation of my manually-deserialized function doesn't work: y is non-const in global scope and is not captured by f; it works as I'd hoped if I do
f = let
y = 3
x -> x + y
end
what's troubling me is that somehow when you remotecall f itself, y gets defined as a global on the worker, so that the second time I deserialize and invoke f on teh remote worker, it succeeds.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the supplied Julia 1.9.2 MWE and compare the serialize/deserialize path with direct remotecall of the closure. Trace the Distributed closure-transfer and Serialization entry points involved in both calls. Done means the differing global-variable behavior is explained and the two paths behave consistently or the intended distinction is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100