Replace custom profiler with `py-spy`
Open
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Currently, the custom profiler causes issues on CI (e.g. https://github.com/dask/distributed/runs/7235536612?check_suite_focus=true#step:11:442) and might not produce completely accurate results (#6499). At the same time, there are great profilers which support standard output formats such as [py-spy](https://github.com/benfred/py-spy). I suggest that we remove the custom profiler in favor of a `py-spy`-based alternative. I understand that @gjoseph92 already has some experience with using `py-spy` to profile `distributed` (https://github.com/gjoseph92/scheduler-profilers).
Contributor guide
Assessment
This issue has not been assessed yet.