JuliaParallel / JuliaParallel/DistributedArrays.jl
Issue with OffsetArrays
- 主要语言
- Julia
- 星标
- 205
- 派生
- 34
- PR 合并指标
- 30 天内没有已合并 PR
描述
I was getting nervous about the communication costs of being imprecise with communications outside of the localpart of the array, so I figured I'd build a halo exchange system by combining OffsetArrays and DistributedArrays.
I intended for that to look like the following, where each array has a 4x4 block of purely local data and a surrounding 1-cell halo to cache remote data pulled using DistributedArrays.
```julia
@everywhere using DistributedArrays
@everywhere using OffsetArrays
r1 = @spawnat 2 OffsetArray(zeros(6,6), 0:5, 0:5)
r2 = @spawnat 3 OffsetArray(zeros(6,6), 0:5, 0:5)
r3 = @spawnat 4 OffsetArray(zeros(6,6), 0:5, 0:5)
r4 = @spawnat 5 OffsetArray(zeros(6,6), 0:5, 0:5)
ras = [r1 r2; r3 r4]
D = DArray(ras)
```
Unfortunately, when I try to run this, I get:
```
julia> D = DArray(ras)
ERROR: MethodError: no method matching OffsetArray{Float64,2,Array{Float64,2}}(::Array{Float64,2})
Closest candidates are:
OffsetArray{Float64,2,Array{Float64,2}}(::Any, ::Any) where {T, N, AA<:AbstractArray} at /home/rick/.julia/packages/OffsetArrays/ICRTb/src/OffsetArrays.jl:10
Stacktrace:
[1] empty_localpart(::Type, ::Int64, ::Type) at /home/rick/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:66
[2] macro expansion at ./task.jl:264 [inlined]
[3] macro expansion at /home/rick/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:84 [inlined]
[4] macro expansion at ./task.jl:244 [inlined]
[5] DArray(::Tuple{Int64,Int64}, ::Array{Future,2}, ::Tuple{Int64,Int64}, ::Array{Int64,2}, ::Array{Tuple{UnitRange{Int64},UnitRange{Int64}},2}, ::Array{Array{Int64,1},1}) at /home/rick/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:82
[6] macro expansion at ./task.jl:266 [inlined]
[7] macro expansion at /home/rick/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:194 [inlined]
[8] macro expansion at ./task.jl:244 [inlined]
[9] DArray(::Array{Future,2}) at /home/rick/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:192
[10] top-level scope at none:0
```
which doesn't seem like an issue with OffsetArrays because using SparseArrays in an analogous way produces no problems:
```julia
@everywhere using DistributedArrays
@everywhere using SparseArrays
r1 = @spawnat 2 sparse(zeros(6,6))
r2 = @spawnat 3 sparse(zeros(6,6))
r3 = @spawnat 4 sparse(zeros(6,6))
r4 = @spawnat 5 sparse(zeros(6,6))
ras = [r1 r2; r3 r4]
D = DArray(ras)
```
Sadly, I'm not familiar enough with Julia to know why the interaction of OffsetArrays and DistributedArrays is problematic.
I've got:
```
[aaf54ef3] DistributedArrays v0.5.1
[6fe1bfb0] OffsetArrays v0.8.1
```
贡献指南
这个仓库没有索引到贡献指南
调研方向
先从失败的 OffsetArray 和 SparseArrays 复现开始,然后阅读 DistributedArrays/src/darray.jl 中 empty_localpart 附近的代码,以及堆栈跟踪中显示的 DArray 构造函数。确定这种交互应当受到支持,还是应记录为不兼容,并添加一个能够记录预期行为的回归测试或文档。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- julia
- 领域
- distributed-systems
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 需要澄清
- 新手友好度
- 35/100