dask / dask/distributed

Feature Request: More fully supported actors

Open
#6,249 3 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

According to the Dask documentation Actors have the following limitations.

> BaseActorFuture are similar to normal Dask Future objects, but not as fully featured. They curently only support the result method and nothing else. They don’t currently work with any other Dask functions that expect futures, like as_completed, wait, or client.gather. They can’t be placed into additional submit or map calls to form dependencies. They communicate their results immediately (rather than waiting for result to be called) and cache the result on the future itself.

> No Resilience: No effort is made to make actor workloads resilient to worker failure. If the worker dies while holding an actor that actor is lost forever.
No Diagnostics: Because the scheduler is not informed about actor computations no diagnostics are available about these computations.
No Load balancing: Actors are allocated onto workers evenly, without serious consideration given to avoiding communication.
Experimental: Actors are a new feature and subject to change without warning

> Currently workers have only a single thread for actors, but this may change in the future.
The result is sent back immediately to the calling side, and is not stored on the worker with the actor. It is cached on the BaseActorFuture object.

The first point limits ease of use of the abstraction (especially being unable to pass the futures around to form dependencies), the second limits deployability in a production environment, and the last obviously impacts performance. I know personally it would be very exciting to see these points improved (probably in that order), and would increase use of Dask outside data science workloads.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.