JuliaLang / JuliaLang/Distributed.jl

Could `@everywhere` support using macros from packages that are loaded in the same block?

Open
#83 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
55
Forks
19
PR merge metrics
No merged PRs in 30d

Description

Consider the following (with TimeZones.jl installed in global enviroment),

using Distributed
addprocs(1)
@everywhere begin
    using TimeZones
    foo() = tz"America/New_York"
end

This should be valid as far as I know.

This failed in julia 1.6, but i think i first noticed this many many versiions ago

julia> using Distributed

julia> addprocs(1)
1-element Vector{Int64}:
 2

julia> @everywhere begin
           using TimeZones
           foo() = tz"America/New_York"
       end
ERROR: On worker 2:
LoadError: UndefVarError: @tz_str not defined
Stacktrace:
 [1] top-level scope
   @ :0
 [2] eval
   @ ./boot.jl:360
 [3] JuliaLang/julia#103
   @ /usr/local/src/julia/julia-1.6/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:274
 [4] run_work_thunk
   @ /usr/local/src/julia/julia-1.6/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:63
 [5] run_work_thunk
   @ /usr/local/src/julia/julia-1.6/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:72
 [6] JuliaLang/julia#96
   @ ./task.jl:411
in expression starting at REPL[3]:3
Stacktrace:
 [1] sync_end(c::Channel{Any})
   @ Base ./task.jl:369
 [2] macro expansion
   @ ./task.jl:388 [inlined]
 [3] remotecall_eval(m::Module, procs::Vector{Int64}, ex::Expr)
   @ Distributed /usr/local/src/julia/julia-1.6/usr/share/julia/stdlib/v1.6/Distributed/src/macros.jl:223
 [4] top-level scope
   @ /usr/local/src/julia/julia-1.6/usr/share/julia/stdlib/v1.6/Distributed/src/macros.jl:207

In contrast

@everywhere using TimeZones
@everywhere foo() = tz"America/New_York"

works just fine.

Similar failure occurs with normal macros, e.g.

@everywhere begin
       using JuMP
       @variable(Model(), x)
end

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Distributed/src/macros.jl, especially the @everywhere and remotecall_eval entry points shown in the stack trace, and reproduce the TimeZones and JuMP examples. Done means a package loaded earlier in the same @everywhere block makes its macros available to later expressions on workers, with tests covering both examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.