dask / dask/distributed

Add metrics for internal transfers

Open
#7,308 0 comments 0 reactions 0 assignees View on GitHub
diagnostics
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

When looking at prometheus metrics it would be helpful to see how much transfer is going on. We have overall network traffic, but this includes communication with external services, like S3.

We already track network transfers internally with `Worker.transfer_{incoming,outgoing}_bytes`, but this is ephemeral and accounts for the amount currently in progress. One way to handle the total would be to add new `Worker.transfer_{incoming,outgoing}_bytes_total` values, and only add to them, rather than also subtract from them.

cc @fjetter , thoughts?

cc @ntabris @dchudz

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.