JuliaLang / JuliaLang/Distributed.jl

Surprising and undocumented `import` handling in `Distributed.@everywhere`

Open
#65 2 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
55
Forks
19
PR merge metrics
No merged PRs in 30d

Description

Consider the following code:

    using Distributed
    addprocs(1)
    @everywhere begin
          # assume MyModule is defined in the path given
          push!(LOAD_PATH, "/path/to/my/module")
          import MyModule
    end

This fails with a LoadError, because @everywhere cherry-picks the imports for each everywhere block and runs them first regardless of the order of statements inside the block. Instead, one has to do this:

    # (...)
    @everywhere push!(LOAD_PATH, "/path/to/my/module")
    @everywhere import MyModule

This behaviour thoroughly breaks my internal parser, because these two blocks look equivalent (modulo an unnecessary synchronization point in the middle). Worse, the behaviour is not documented, and in fact contradicts the documentation.

Unfortunately, no rationale is given in the code as to why this was chosen. However, I think at the very least, one should amend the documentation with a corresponding note and add a rationale for this in the code.

One could maybe also issue a warning for @everywhere blocks which do not have import statements as their first statements. This would help poor newbies like me :)

Julia version: 1.2.0 (also present in master)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read stdlib/Distributed/src/macros.jl around the linked lines 189 and 197, then reproduce the two @everywhere examples from the issue. Document the import-order behavior and add the requested code rationale; treat a warning for later statements as optional unless the project defines its scope. Done means the documentation no longer contradicts the observed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
distributed-systems
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.