JuliaParallel / JuliaParallel/DistributedArrays.jl
Allowing a DArray to do as standard Array does
- Lenguaje dominante
- Julia
- Estrellas
- 205
- Forks
- 34
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
Dear All,
I'm considering the following MWE, which consists of creating a matrix and vector of a `struct` an doing a calculation on them. I import the relevant mathematical operators from base, and it just works. Next I `addprocs` and try it with `DArray`, and the same calculation fails. Is there a way to enable this kind of functionality? It's definitely possible that I'm barking up the wrong tree and there's a much better way of achieving this - please let me know!
```
using Distributed
import Base.+, Base.*, Base.zero
let
struct Foo{T<:Number}
x::T
end
Base.:+(a::Foo, b::Foo) = Foo(a.x + b.x)
Base.:*(a::Foo, b::Foo) = Foo(a.x * b.x)
Base.:zero(a::Foo{T}) where {T} = Foo(zero(T))
function runlocal()
A = [Foo(rand()) for i in 1:3, j in 1:3]
b = [Foo(rand()) for j in 1:3]
@show A * b
end
runlocal()
end
addprocs(2)
@everywhere using DistributedArrays
@everywhere let
struct Foo{T<:Number}
x::T
end
Base.:+(a::Foo, b::Foo) = Foo(a.x + b.x)
Base.:*(a::Foo, b::Foo) = Foo(a.x * b.x)
Base.:zero(a::Foo{T}) where {T} = Foo(zero(T))
function rundistributed()
A = @DArray [Foo(rand()) for i in 1:3, j in 1:3]
b = @DArray [Foo(rand()) for j in 1:3]
@show A * b
end
rundistributed()
end
```
Thanks,
James
Output:
```
ERROR: On worker 2:
MethodError: Cannot `convert` an object of type Int64 to an object of type Foo{Float64}
Closest candidates are:
convert(::Type{T}, ::T) where T at essentials.jl:154
Foo{Float64}(::Any) where T<:Number at REPL[6]:3 (method too new to be called from this world context.)
fill! at ./array.jl:283
#253 at /home/cookj/.julia/packages/DistributedArrays/f3HAT/src/linalg.jl:111
run_work_thunk at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:56
#remotecall_fetch#148 at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:364
remotecall_fetch at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:364
#remotecall_fetch#152 at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406
remotecall_fetch at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406 [inlined]
#252 at /home/cookj/.julia/packages/DistributedArrays/f3HAT/src/linalg.jl:107 [inlined]
#672 at ./asyncmap.jl:100
macro expansion at ./asyncmap.jl:235 [inlined]
#688 at ./task.jl:259
#remotecall_wait#154(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:421
remotecall_wait(::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:412
#remotecall_wait#157(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:433
remotecall_wait(::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:433
(::getfield(Distributed, Symbol("##163#165")){Module,Expr})() at ./task.jl:259
...and 2 more exception(s).
Stacktrace:
[1] sync_end(::Array{Any,1}) at ./task.jl:226
[2] macro expansion at ./task.jl:245 [inlined]
[3] remotecall_eval(::Module, ::Array{Int64,1}, ::Expr) at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/macros.jl:206
[4] top-level scope at /home/cookj/builds/julia/usr/share/julia/stdlib/v1.0/Distributed/src/macros.jl:190
```
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Línea de trabajo
Start with the MWE and the distributed matrix-vector path in src/linalg.jl, especially lines 107 and 111 named by the stack trace. Reproduce it with Julia workers and compare the DArray operation with the local Array case; done means the custom Foo calculation no longer fails on workers.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- julia
- Área
- distributed-systems
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 35/100