An API for interegating recorded data
- Dominant language
- Python
- Stars
- 15.2k
- Forks
- 461
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 10
Description
### Is there an existing proposal for this?
- [X] I have searched the existing proposals
### Is your feature request related to a problem?
I enjoy the flamegraph implementations, etc meant for human eyes. What I would love to see is an API to programmatically pull out the desired metrics of potentially massive amounts of runs (different inputs, etc)
### Describe the solution you'd like
I would love if Memray had functions to pull out specific useful data, like peak memory for various dimentions
### Alternatives you considered
My current workaround (very hacky, but all I could figure out due to lack of options):
```
import subprocess
import json
import re
from bs4 import BeautifulSoup
from pathlib import Path
import numpy as np
def pad_arg(arg: int, leading_zeros: int = 6):
return str(arg).zfill(leading_zeros)
peak_memory_matrix_raw = []
peak_memory_series_x = []
peak_memory_series_y = []
y_axis = range(1000, 11000, 1000)
x_axis = range(100, 1100, 100)
for row in y_axis:
peak_memory_row_raw = []
for col in x_axis:
subprocess.run([f"memray flamegraph analysis/mem-pivot-{pad_arg(row)}-{pad_arg(col)}.bin"], shell=True)
with Path(f"analysis/memray-flamegraph-mem-pivot-{pad_arg(row)}-{pad_arg(col)}.html").open('r') as html:
soup = BeautifulSoup(html)
script = soup.find('script', {'type': 'text/javascript'})
memory_records = json.loads(script.contents[0].strip().split('const memory_records = ')[1].split(';')[0])
peak_memory = max([memory_record[1] for memory_record in memory_records])
peak_memory_row_raw.append(peak_memory)
peak_memory_series_x.append(row * col)
peak_memory_series_y.append(peak_memory)
peak_memory_matrix_raw.append(peak_memory_row_raw)
peak_memory_matrix = np.matrix(peak_memory_matrix_raw)
```
Contributor guide
Research direction
Start by reviewing the existing memray flamegraph command and the generated HTML data, especially the memory_records values extracted in the example. Define which recorded metrics and dimensions the API must expose, then verify that the requested data can be retrieved programmatically without HTML scraping.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devtools, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100