bloomberg / bloomberg/memray

Implement plot of module usage over time

Open
#219 1 comment 7 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
15.2k
Forks
461
Avg merge
5d 3h
Merged PRs (30d)
10

Description

One of the aspects that makes `memray` different from other similar tools is that we capture a ton of information from the running process. Currently, we mainly leverage this information to generate reports that happen at the process high water mark. This is likely the most interesting point for reporting, but we are not using a lot of the information that we collect. This information can be very useful to provide a different view of the program: how memory evolves over time. In this regard, we already offer total RSS and HEAP allocation over time but it would be great if we could break down that total in different pieces.

After some discussion with @godlygeek, we plan to experiment with this and start offering a way to break down the total usage by module. Here is a **rough** example of what that would look like:

![image](https://user-images.githubusercontent.com/11718525/192781619-f3ed91a2-ad1e-4c2f-a27e-bfaf1f3921b5.png)

Ideally, we can break down the chart in different ways:

* By module
* By function
* By file

As we don't want to collect timing information per allocation because that will increase the file size by a ridiculous amount (8 bytes or so per allocation) we will make time snapshots every time we emit one of the RSS ticks in the background thread. Is likely that we need to collapse those in a broader granularity because that's likely still too much.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing Memray's existing total RSS and heap allocation reports and the background-thread RSS ticks that provide time snapshots. Define a focused first scope for breaking usage down by module, then determine how snapshot granularity and the chart should represent the data; done means a working time-based breakdown matching the intended reporting view.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.