JuliaLang / JuliaLang/Distributed.jl

include on workers has somewhat strange behavior

Open
#24 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
55
Forks
19
PR merge metrics
No merged PRs in 30d

Description

If I'm reading include_from_node1 correctly, we convert path passed to include to an absolute path on the current worker, and then fetch the source from node 1. This makes include inside packages work, since the absolute path is relative to the path to the required file on node 1. However, this behavior seems more questionable for a bare include not inside a file that was reloaded or required (e.g. @spawnat 2 include("mycode.jl")), since in this case the absolute path is relative to the worker's current working directory, which might not exist on node 1. In that case, I think we should either:

  • include relative to the current working directory on node 1. (This is not so weird, since require and reload search the current working directory on node 1.)
  • include the file from the local file system. But this may be ill-advised. If you include a local file and it calls include, that should naturally refer to a local file. But if you include a local file and it calls require or using, do you load local code or code from node 1?
  • Throw some kind of error.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with include_from_node1 in base/loading.jl at the linked lines, then trace how bare include behaves under @spawnat and how package-loaded files are resolved. Compare the current-worker and node-1 path cases described in the issue. Done means selecting and implementing one consistent resolution or error behavior, with its interaction with nested include, require, and using made explicit.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.