JuliaParallel / JuliaParallel/DistributedArrays.jl

DArray should be lazy/futures

未关闭
#209 2 条评论 0 个 reaction 已指派 1 人 已被 @vchuravy 认领 在 GitHub 查看
enhancement help wanted speculative
主要语言
Julia
星标
205
派生
34
PR 合并指标
30 天内没有已合并 PR

描述

One of the current issues with DArray is that each operation is immediately synchronizing. Requiring a distributed operation to finish before we can carry on with scheduling new operations. This simplifies the design,
but limits the scalability. Ideally we would want operations on DArray to be async/lazy similar to how CuArray works, and only synchronize on `show` and `convert`.

The major design issue here is to guarantee consistency. Operations need to appear to have executed in-order,
even though we might want to be able to execute reads out-of-order, but we will have to deal with updating data in-place. One idea might be to use vector clocks or look into how Fractal handles this or to run a consensus protocol to establish operations that can commit.

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。