JuliaParallel / JuliaParallel/Dagger.jl

Using shards inside spawn_datadeps leads to generic memory error

Open
#665 3 comments 0 reactions 0 assignees View on GitHub
bug datadeps
Dominant language
Julia
Stars
723
Forks
90
Avg merge
1d 37m
Merged PRs (30d)
9

Description

Using a shard inside a spawn_datadeps environment leads to the error `ERROR: Type GenericMemory does not have a definite size.`

A simple example of this is
```
using Distributed; addprocs(2)
nworkers = length(workers())
@everywhere using Dagger
@everywhere using LinearAlgebra

A = rand(Blocks(8, 4), 4*4, 4)
B = rand(Blocks(2), 4)
C = Dagger.@shard rand(4*4)

Dagger.spawn_datadeps() do

for (idx, w) in enumerate(workers())
Dagger.@spawn scope=Dagger.scope(worker=w) mul!(Out(C), A.chunks[idx], B.chunks[idx])
end

# further operations that use C
end
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.