Task state "alias" for executing tasks
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
There is a common misconception that the scheduler doesn't truly know if a task is executing on the worker or not. While this is certainly not accurate information, it is not entirely true since the worker does submit the tasks that are currently on the thread pool as part of the heartbeat, see
https://github.com/dask/distributed/blob/40456b46598a1b096a580af6ad60c7bb6852a3b1/distributed/worker.py#L1242-L1246
This information is actually mostly unused. The submitted timing information is merely used to update unknown prefix timings, see
https://github.com/dask/distributed/blob/40456b46598a1b096a580af6ad60c7bb6852a3b1/distributed/scheduler.py#L4032-L4039
I'm a bit hesitant of introducing this information as part of scheduling heuristics but in terms of observability this _may_ be interesting and would remove a certain user confusion allowing us disambiguate the state `processing` into `assigned` and `executing` on the dashboard, etc. (The dashboard doesn't really care about heartbeat delays, scheduling does. Hence why I don't really want to incorporate this into scheduling if not truly necessary)
As such, I wonder if we wanted to introduce a no-op/alias state "executing" which would be identical to "processing" for all scheduling matters but may be visualized a bit more explicitly.
Thoughts? @gjoseph92 @crusaderky ?
Contributor guide
Assessment
This issue has not been assessed yet.