bloomberg / bloomberg/memray

An API for interegating recorded data

Open
#637 2 comments 3 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
15.2k
Forks
461
Avg merge
5d 3h
Merged PRs (30d)
10

Description

### Is there an existing proposal for this?

- [X] I have searched the existing proposals

### Is your feature request related to a problem?

I enjoy the flamegraph implementations, etc meant for human eyes. What I would love to see is an API to programmatically pull out the desired metrics of potentially massive amounts of runs (different inputs, etc)

### Describe the solution you'd like

I would love if Memray had functions to pull out specific useful data, like peak memory for various dimentions

### Alternatives you considered

My current workaround (very hacky, but all I could figure out due to lack of options):

```
import subprocess
import json
import re
from bs4 import BeautifulSoup
from pathlib import Path
import numpy as np

def pad_arg(arg: int, leading_zeros: int = 6):
return str(arg).zfill(leading_zeros)

peak_memory_matrix_raw = []
peak_memory_series_x = []
peak_memory_series_y = []
y_axis = range(1000, 11000, 1000)
x_axis = range(100, 1100, 100)
for row in y_axis:
peak_memory_row_raw = []
for col in x_axis:
subprocess.run([f"memray flamegraph analysis/mem-pivot-{pad_arg(row)}-{pad_arg(col)}.bin"], shell=True)
with Path(f"analysis/memray-flamegraph-mem-pivot-{pad_arg(row)}-{pad_arg(col)}.html").open('r') as html:
soup = BeautifulSoup(html)
script = soup.find('script', {'type': 'text/javascript'})
memory_records = json.loads(script.contents[0].strip().split('const memory_records = ')[1].split(';')[0])
peak_memory = max([memory_record[1] for memory_record in memory_records])
peak_memory_row_raw.append(peak_memory)
peak_memory_series_x.append(row * col)
peak_memory_series_y.append(peak_memory)
peak_memory_matrix_raw.append(peak_memory_row_raw)
peak_memory_matrix = np.matrix(peak_memory_matrix_raw)
```

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing memray flamegraph command and the generated HTML data, especially the memory_records values extracted in the example. Define which recorded metrics and dimensions the API must expose, then verify that the requested data can be retrieved programmatically without HTML scraping.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.