JuliaParallel / JuliaParallel/DistributedArrays.jl

Allowing a DArray to do as standard Array does

未关闭
#202 4 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Julia
星标
205
派生
34
PR 合并指标
30 天内没有已合并 PR

描述

Dear All,

I'm considering the following MWE, which consists of creating a matrix and vector of a `struct` an doing a calculation on them. I import the relevant mathematical operators from base, and it just works. Next I `addprocs` and try it with `DArray`, and the same calculation fails. Is there a way to enable this kind of functionality? It's definitely possible that I'm barking up the wrong tree and there's a much better way of achieving this - please let me know!

```
using Distributed

import Base.+, Base.*, Base.zero

let
struct Foo{T<:Number}
x::T
end
Base.:+(a::Foo, b::Foo) = Foo(a.x + b.x)
Base.:*(a::Foo, b::Foo) = Foo(a.x * b.x)
Base.:zero(a::Foo{T}) where {T} = Foo(zero(T))
function runlocal()
A = [Foo(rand()) for i in 1:3, j in 1:3]
b = [Foo(rand()) for j in 1:3]
@show A * b
end
runlocal()
end

addprocs(2)
@everywhere using DistributedArrays
@everywhere let
struct Foo{T<:Number}
x::T
end
Base.:+(a::Foo, b::Foo) = Foo(a.x + b.x)
Base.:*(a::Foo, b::Foo) = Foo(a.x * b.x)
Base.:zero(a::Foo{T}) where {T} = Foo(zero(T))
function rundistributed()
A = @DArray [Foo(rand()) for i in 1:3, j in 1:3]
b = @DArray [Foo(rand()) for j in 1:3]
@show A * b
end
rundistributed()
end
```

Thanks,
James

Output:

```
ERROR: On worker 2:
MethodError: Cannot `convert` an object of type Int64 to an object of type Foo{Float64}
Closest candidates are:
convert(::Type{T}, ::T) where T at essentials.jl:154
Foo{Float64}(::Any) where T<:Number at REPL[6]:3 (method too new to be called from this world context.)
fill! at ./array.jl:283
#253 at /home/cookj/.julia/packages/DistributedArrays/f3HAT/src/linalg.jl:111
run_work_thunk at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:56
#remotecall_fetch#148 at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:364
remotecall_fetch at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:364
#remotecall_fetch#152 at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406
remotecall_fetch at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406 [inlined]
#252 at /home/cookj/.julia/packages/DistributedArrays/f3HAT/src/linalg.jl:107 [inlined]
#672 at ./asyncmap.jl:100
macro expansion at ./asyncmap.jl:235 [inlined]
#688 at ./task.jl:259
#remotecall_wait#154(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:421
remotecall_wait(::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:412
#remotecall_wait#157(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:433
remotecall_wait(::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:433
(::getfield(Distributed, Symbol("##163#165")){Module,Expr})() at ./task.jl:259

...and 2 more exception(s).

Stacktrace:
[1] sync_end(::Array{Any,1}) at ./task.jl:226
[2] macro expansion at ./task.jl:245 [inlined]
[3] remotecall_eval(::Module, ::Array{Int64,1}, ::Expr) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/macros.jl:206
[4] top-level scope at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/macros.jl:190
```

贡献指南

这个仓库没有索引到贡献指南

调研方向

Start with the MWE and the distributed matrix-vector path in src/linalg.jl, especially lines 107 and 111 named by the stack trace. Reproduce it with Julia workers and compare the DArray operation with the local Array case; done means the custom Foo calculation no longer fails on workers.

由索引模型根据 Issue 内容生成。

评估

技术栈
julia
领域
distributed-systems
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。