dask / dask/distributed

spark-style accumulators

Open
#1,582 2 comments 1 reaction 0 assignees View on GitHub
discussion enhancement
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

It would be easy to modify/derive from the existing Variables to become race-safe counters or sum objects like Sparks accumulators. The only `set` method would be on initialization, thereafter only `get` and `add`-like methods would be allowed. The latter could be made syntactically nice with in-place add.

I'm not sure how much of a demand there is for this, but it seems useful.

Note that we would probably not be able to protect against tasks that run more than once for some reason, i.e., the add method should be considered a side-effect.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.