JuliaParallel / JuliaParallel/Dagger.jl

Slowdown in scheduler after extended eager api usage

Open
#283 2 comments 0 reactions 0 assignees View on GitHub
bug eager api performance
Dominant language
Julia
Stars
723
Forks
90
Avg merge
1d 37m
Merged PRs (30d)
9

Description

This piece of code (if it doesn't hang) slows down significantly at ~1800 iterations for me.
From observing the threads usage it seems like the main thread is constantly occupied and the time period between each spawn increases every iteration (it can be observed through cpu usage on threads).

Most likely related to the huge number of thunks generated in this code over time

```julia
julia> using Dagger

julia> c = Dagger.@spawn 10+10; b = (x) -> x + 10; a = (x) -> x .+ fetch.([Dagger.spawn(b, x + i) for i in 1:100]);

julia> for i in 1:10000
r = Dagger.@spawn a(c)
fetch(r); println(i)
end
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.