Restrict task/actor placement based on free worker memory
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
First off, thank you all for this great framework!
I ran into the following issue:
I want to distribute actors which consume a significant amount of RAM. For instance, each worker holds a large NumPy array ~ 1GB in memory, thus considering available worker memory is crucial for actor placement.
Furthermore, let's assume the following workers:
- worker 1: 1.5 GB memory limit
- worker 2: 3.5 GB memory limit
Hence, worker 1 can hold a maximum of 1 actor; worker 2 can hold a maximum of 3 actors (ignoring any spill, pause, terminate for now).
Unfortunately, `resources={...}` are only considered while processing a task or actor. Therefore, submitting 4 actors results in the following **invalid** placement:
- worker 1: 2 actors
- worker 2: 2 actors
As a result, worker 1 will die due to memory-limit violation. So I asked myself if there is any possibility to take available worker memory into account when assigning an actor to a worker. If there is no solution yet, I'd love to open a PR.
Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.