dask / dask/distributed

Fine performance metrics: seceded tasks

Open
#7,675 0 comments 0 reactions 0 assignees View on GitHub
diagnostics
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

- Part of #7665

As of #7586, seceded tasks don't receive any special treatment. This means that, when there are seceded tasks, the sum of `Worker.digests_total["execute", *, *, "seconds"]` could exceed the number of threads * worker uptime.

Add special consideration for seceded and rejoined tasks, so that only the time that is spent while not seceded goes into `Worker.digests_total["execute", *, *, "seconds"]`.

### Bonus deliverable
A CPU-heavy seceded task can cause regular tasks to exhibit a very high `thread-noncpu` time.
Measure CPU usage (`time.thread_time()`) while seceded and log it somewhere separate.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.