JuliaLang / JuliaLang/Distributed.jl
Order dependent module loading with `Distributed`
Open
@vchuravy is already working on this.
Since Aug 24, 2018.
bug
- Dominant language
- Julia
- Stars
- 55
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Description
Discovered by @ararslan in his quest to get Nanosoldier back online.
In distributed mode using X should make X available on the worker nodes as a root module so that remotecall(X.f) works without a @everywhere X.
This correctly works:
julia> using Distributed
julia> addprocs(2)
2-element Array{Int64,1}:
2
3
julia> @everywhere begin
function log_require(mod::Base.PkgId)
@info "Loading pkg $mod $(Base.root_module_exists(mod)) on proc $(myid())"
end
push!(Base.package_callbacks, log_require)
end
julia> using BenchmarkTools
[ Info: Loading pkg JSON [682c06a0-de6a-54ab-a142-c8b1cf79cde6] true on proc 1
[ Info: Loading pkg JSON [682c06a0-de6a-54ab-a142-c8b1cf79cde6] true on proc 2
[ Info: Loading pkg JSON [682c06a0-de6a-54ab-a142-c8b1cf79cde6] true on proc 3
[ Info: Loading pkg BenchmarkTools [6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf] true on proc 2
[ Info: Loading pkg BenchmarkTools [6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf] true on proc 3
[ Info: Loading pkg BenchmarkTools [6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf] true on proc 1
If the user says using X before addprocs we no longer trigger the callback on a subsequent using X,
thereby triggering https://github.com/JuliaLang/julia/pull/28857#issuecomment-415590074
julia> using Distributed
julia> using BenchmarkTools
julia> addprocs(2)
2-element Array{Int64,1}:
2
3
julia> @everywhere begin
function log_require(mod::Base.PkgId)
@info "Loading pkg $mod $(Base.root_module_exists(mod)) on proc $(myid())"
end
push!(Base.package_callbacks, log_require)
end
julia> using BenchmarkTools
julia> @everywhere using BenchmarkTools
[ Info: Loading pkg JSON [682c06a0-de6a-54ab-a142-c8b1cf79cde6] true on proc 2
[ Info: Loading pkg JSON [682c06a0-de6a-54ab-a142-c8b1cf79cde6] true on proc 3
[ Info: Loading pkg BenchmarkTools [6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf] true on proc 2
[ Info: Loading pkg BenchmarkTools [6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf] true on proc 3
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.