bloomberg / bloomberg/memray

Add --split-threads support to memray transform csv

Open
#996 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
15.2k
Forks
461
Avg merge
5d 3h
Merged PRs (30d)
10

Description

### Is there an existing proposal for this?

- [x] I have searched the existing proposals

### Is your feature request related to a problem?

`memray transform csv` includes `tid` and `thread_name` columns in its output schema, but the transform command currently retrieves allocation records with threads merged.

When allocations with the same stack occur on multiple threads, the exported record can therefore contain `tid = -1` and `thread_name = "merged thread"` instead of preserving the original thread identities.

This makes the CSV output difficult to use for per-thread analysis of multi-threaded Python or C++ workloads. It also differs from reporters such as flamegraph and table, which expose a `--split-threads` option.

### Describe the solution you'd like

Add a `--split-threads` option to `memray transform`, initially at least for the CSV format.

When enabled for CSV output, the command would retrieve allocation records with `merge_threads=False` and allow `TransformReporter` to process those records. The existing `tid` and `thread_name` columns would then preserve the original thread information.

The default behavior would remain unchanged.

Would you prefer `--split-threads` to be:

1. supported for CSV only, since its schema already represents thread identity; or
2. supported across transform formats, with Speedscope potentially emitting separate profiles for each thread?

I would be happy to implement the narrower CSV-only version first, including CLI integration tests and reporter tests, if that is the preferred scope.

### Alternatives you considered

Users can inspect per-thread information through the HTML reporters, but those reports do not provide a convenient machine-readable export for downstream analysis.

Contributor guide

Open the contributing guide

Research direction

Start with the memray transform command and TransformReporter, then inspect how existing flamegraph and table reporters implement --split-threads. Run the CLI integration and reporter tests mentioned in the issue; done means CSV can preserve thread identities when enabled while the default behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.