bloomberg / bloomberg/memray

Move the runner into a separate package with minimal dependencies

Open
#557 10 comments 0 reactions 0 assignees View on GitHub
enhancement good first issue Packaging
Dominant language
Python
Stars
15.2k
Forks
461
Avg merge
5d 3h
Merged PRs (30d)
10

Description

### Is there an existing proposal for this?

- [X] I have searched the existing proposals

### Is your feature request related to a problem?

The system I am currently working on involves some confidential customer data to which regular developers do not have access. Due to the complex nature of ML algorithms and libraries involved, sometimes we encounter anomalous memory usage which we cannot easily reproduce with our test data set. I am considering integration of some memory profiling mechanism, possibly involving memray which would be selectively enabled in production environment to help us to get insights into problematic cases without requiring access to customer data.

For multiple reasons (security, reliability, docker image size to name a few) we try to minimize the number of our dependencies. Memray has a number of dependencies which are unnecessary for trace collection:

https://github.com/bloomberg/memray/blob/578e02d20bc63bddc0fc25c28bf7569519b89838/setup.py#L91-L96

and those have their own dependencies as well.

### Describe the solution you'd like

It would be great to split memray into two packages, tentatively named `memray-core` and `memray`. The former would only be able to perform collection, and the later would do everything memray does today.

### Alternatives you considered

Another possibility would be having all the dependencies not related to `memray.Tracker` to be under "extras" but that would probably be a nuisance for most ordinary users who would want a simple `pip install memray` to install everything.

Contributor guide

Open the contributing guide

Research direction

Start with the dependency declarations in setup.py and the memray.Tracker entry point. Review how the proposed memray-core and memray packages would separate collection from the rest of the functionality, including the existing full-install behavior. Done means the package boundary and dependency split are defined without requiring collection users to install unrelated dependencies.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.