dask / dask/distributed

Replace custom profiler with `py-spy`

Open
#6,800 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Currently, the custom profiler causes issues on CI (e.g. https://github.com/dask/distributed/runs/7235536612?check_suite_focus=true#step:11:442) and might not produce completely accurate results (#6499). At the same time, there are great profilers which support standard output formats such as [py-spy](https://github.com/benfred/py-spy). I suggest that we remove the custom profiler in favor of a `py-spy`-based alternative. I understand that @gjoseph92 already has some experience with using `py-spy` to profile `distributed` (https://github.com/gjoseph92/scheduler-profilers).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.