Feature Request: More fully supported actors
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
According to the Dask documentation Actors have the following limitations.
> BaseActorFuture are similar to normal Dask Future objects, but not as fully featured. They curently only support the result method and nothing else. They don’t currently work with any other Dask functions that expect futures, like as_completed, wait, or client.gather. They can’t be placed into additional submit or map calls to form dependencies. They communicate their results immediately (rather than waiting for result to be called) and cache the result on the future itself.
> No Resilience: No effort is made to make actor workloads resilient to worker failure. If the worker dies while holding an actor that actor is lost forever.
No Diagnostics: Because the scheduler is not informed about actor computations no diagnostics are available about these computations.
No Load balancing: Actors are allocated onto workers evenly, without serious consideration given to avoiding communication.
Experimental: Actors are a new feature and subject to change without warning
> Currently workers have only a single thread for actors, but this may change in the future.
The result is sent back immediately to the calling side, and is not stored on the worker with the actor. It is cached on the BaseActorFuture object.
The first point limits ease of use of the abstraction (especially being unable to pass the futures around to form dependencies), the second limits deployability in a production environment, and the last obviously impacts performance. I know personally it would be very exciting to see these points improved (probably in that order), and would increase use of Dask outside data science workloads.
Contributor guide
Assessment
This issue has not been assessed yet.