dask / dask/distributed

Wrong count for the ready tasks from scheduler_info()

Open
#4,437 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Hello,

I've been using the distributed to get current tasks that are either being processed or in the processing queue. I use scheduler_info() to get the workers info and the number of task each of the workers has in their queue.

Based on the documentation, 'executing' shows the number of tasks being processed by a worker, and 'ready' means the number of tasks in each worker that are ready to be processed by any worker. However, these numbers are not always correct.

I have 13 workers and 10 tasks being processed by these workers (I got these values from the web UI). So, it means 3 workers are free and each one of the workers has 1 in the process. When I use the 'scheduler_info()', one of the workers has 10 ready tasks, one of them has 47 ready, and another one has 59 ready tasks. The question is that how are these numbers calculated and are those reliable or not?

Thanks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.