Unresponsive workers should be flagged on the dashboard
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
- Related to #8537
Let's hamstring a worker:
```python
async def kill_event_loop():
while True:
pass
fut = c.submit(kill_event_loop)
```
This makes the worker completely unresponsive. After 5 minutes (`distributed.scheduler.worker-ttl`) without a single heartbeat coming through, the scheduler will disconnect it forcefully.
Until that happens, this is what I'm seeing on the dashboard:

# Desired behaviour
There should be an indication on the dashboard of how many seconds have passed since the last heartbeat.
There should be also be visual cues - e.g. the whole line turning red - to indicate when you get to a significant threshold since the last heartbeat - e.g. max(4x of the expected heartbeat ratio, 25% of the worker-ttl)
Contributor guide
Assessment
This issue has not been assessed yet.