JuliaParallel / JuliaParallel/DistributedArrays.jl

master_slave DArray copy

Open
#132 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
205
Forks
34
PR merge metrics
No merged PRs in 30d

Description

Hello,

First, thank you for a great package!

I have had an issue with `copy` of `DArray` when connected in `master_slave` topology.

```julia
julia> addprocs(2,topology=:master_slave)
2-element Array{Int64,1}:
2
3

julia> nprocs()
3
julia> using DistributedArrays

julia> dd=drand((11,21,2,13),workers(),[1,1,1,2]);

julia> @eval @everywhere println(localindexes($dd))
(1:0,1:0,1:0,1:0)
From worker 2: (1:11,1:21,1:2,1:7)
From worker 3: (1:11,1:21,1:2,8:13)

julia> ddc=copy(dd);
ERROR: On worker 2:
peer 3 is not connected to 2. Topology : master_slave
...
```
I have shortened the error output, surely you can reproduce it.

Is there a reason why this is an error?

Especially that this works...
```julia

julia> dd=drand((11,21,2,13),workers(),[1,1,1,2]);

julia> @everywhere using DistributedArrays

julia> @eval ddc2=DArray(I->copy(localpart($dd)),$dd);

julia> sumabs2(ddc2-dd)
0.0

julia> @eval @everywhere println(localindexes($ddc2))
(1:0,1:0,1:0,1:0)
From worker 2: (1:11,1:21,1:2,1:7)
From worker 3: (1:11,1:21,1:2,8:13)

julia> @eval @everywhere println(localindexes($dd))
(1:0,1:0,1:0,1:0)
From worker 3: (1:11,1:21,1:2,8:13)
From worker 2: (1:11,1:21,1:2,1:7)
```
Finally, is there a reason why `similar` does not yield a `DArray` with the same distribution?

```julia
julia> dd=drand((11,21,2,13),workers(),[1,1,1,2]);

julia> @eval @everywhere println(localindexes($dd))
(1:0,1:0,1:0,1:0)
From worker 2: (1:11,1:21,1:2,1:7)
From worker 3: (1:11,1:21,1:2,8:13)

julia> ddc=similar(dd);

julia> @eval @everywhere println(localindexes($ddc))
(1:0,1:0,1:0,1:0)
From worker 2: (1:11,1:11,1:2,1:13)
From worker 3: (1:11,12:21,1:2,1:13)
```

I appreciate any help on this, and again thanks for the effort with this package!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the master_slave examples using DArray, copy, similar, and localpart as shown in the issue. Inspect the implementations and tests for DArray copy and similar, then verify behavior with the provided localindexes and sumabs2 checks. Done means copy and similar handle the reported distribution or clearly document the intended behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.