JuliaParallel / JuliaParallel/DistributedArrays.jl

DArray should be lazy/futures

Offen
#209 2 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @vchuravy Auf GitHub ansehen
enhancement help wanted speculative
Vorherrschende Sprache
Julia
Sterne
205
Forks
34
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

One of the current issues with DArray is that each operation is immediately synchronizing. Requiring a distributed operation to finish before we can carry on with scheduling new operations. This simplifies the design,
but limits the scalability. Ideally we would want operations on DArray to be async/lazy similar to how CuArray works, and only synchronize on `show` and `convert`.

The major design issue here is to guarantee consistency. Operations need to appear to have executed in-order,
even though we might want to be able to execute reads out-of-order, but we will have to deal with updating data in-place. One idea might be to use vector clocks or look into how Fractal handles this or to run a consensus protocol to establish operations that can commit.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.