bloomberg / bloomberg/pytest-memray

Pytest-memray segfaults where memray does not

Open
#138 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
423
Forks
39
Avg merge
5h 19m
Merged PRs (30d)
1

Description

Hello, I am currently getting a segfault on a private codebase when using `pytest-memray`, but I am NOT getting the same segfault when using only `memray` without the pytest extension to profile python/pytest, i.e. running `pytest --memray ...` causes a segfault but `memray run -m pytest ...` does not.

I initially suspected pytest-memray was not activating greenlet support correctly compared to core memray, as we use the async engine in sqlalchemy and other users have reported segfaults without this support: https://github.com/bloomberg/memray/issues/166. However, I have confirmed via logged warnings that greenlet support is active on both pytest-memray and memray core when running pytest for my codebase. Additionally, the stack trace from the coredump looks a bit related to segfaulting when trying to trace a malloc call than tracing a function like in the linked bug:

```
#0 0x00007f505bc52e2c in ?? () from /lib/x86_64-linux-gnu/libc.so.6

#1 0x00007f505bc03fb2 in raise () from /lib/x86_64-linux-gnu/libc.so.6

#2

#3_PyCode_LineNumberFromArray (index=151147732, co=0x7f4f94022b20) at /usr/local/src/conda/python-3.11.9/Include/internal/pycore_code.h: 556

#4 PyCode_Addr2Line (co=0x7f4f94022620, addrq-302295464) at /usr/local/src/conda/python-3.11.9/0bjects/codeobject.c:773

#5 0x00007f505a2a274e in memray:: tracking_api:: PythonStackTracker::emitPending PushesAndPops() [clone .constprop.0] (this ) at src/memray/_memray/tracking_api.cpp:182

#6 0x00007f505a28f2ad in memray:: tracking_api:: Tracker:: trackAllocationImpl (this=0x55a999bcae10, ptr=0x7f4f94051800, size=528, func=, trace...) at src/memray/_memray/tracking_api.cpp:153

#7 0x00007f505a27b578 in memray:: tracking_api:: Tracker:: trackAllocation (func=memray:: hooks:: Allocator:: MALLOC, size 528, ptr=0x7f4f94051800) at src/memray/_memray/tracking_api.h:228

#8 memray:: intercept::malloc (size=528) at src/memray/_memray/hooks.cpp:177

#9 0x000055a995289924 in PyMem_RawMalloc (size=, ctx=) at /usr/local/src/conda/python-3.11.9/Objects/obmalloc.c:181

#10 PyMem_RawMalloc (size=) at /usr/local/src/conda/python-3.11.9/Objects/obmalloc.c:586

#11_PyObject_Malloc (ctx=, nbytes=) at /usr/local/src/conda/python-3.11.9/Objects/obmalloc.c:2003

#12_PyObject_Malloc (nbytes=528, ctx=) at /usr/local/src/conda/python-3.11.9/Objects/obmalloc.c:1996

#13 PyMem Malloc (size=size@entry=528) at /usr/local/src/conda/python-3.11.9/0bjects/obmalloc.c:623

#14 0x000055a9953a0c78 in newblock (deque deque@entry=0x7f4f9f3cb970) at /usr/local/src/conda/python-3.11.9/Modules/_collectionsmodule.c:132

#15 0x000055a9953a0c0d in deque_new (type=, kwds=)

at /usr/local/src/conda/python-3.11.9/Modules/_collectionsmodule.c:162

#16 0x000055a9952997e4 in type_call (kwds=0x0, args=(), type-0x55a9955e7d20 ) at /usr/local/src/conda/python-3.11.9/Objects/typeobject.c:1

#17_PyObject_MakeTpCall (tstate=0x55a99b1e0140, callable , args=, nargs=0, keywords=0x0) at /usr/local/src/conda/python-3.11.9/Objects/call.c:214

#18 0x000055a9951bd584 in trace_call_function (kwnames-exe, nargs=0, args=, func=, tstate=, frame@entry=0x7f5059884890, throwflag-throwflag@entry=0)

at /usr/local/src/conda/python-3.11.9/Python/ceval.c:4764

#20 0x000055a9952ca981 in _PyEval_EvalFrame (throwflag=0, frame=0x7f5059884890, tstate=0x55a99b1e0140)

at /usr/local/src/conda/python-3.11.9/Include/internal/pycore ceval.h:73
```

Obviously we don't expect a segfault when running memray, but also it's strange that it only happens with pytest-memray and not when using memray core.

Given I've confirmed greenlet is activated in both cases, I'm not sure how to proceed with further debugging the cause of this segfault.

Contributor guide

Open the contributing guide

Research direction

Start by comparing `pytest --memray ...` with `memray run -m pytest ...` using a reproducible case, since the report only reproduces the failure in the pytest integration. Read `src/memray/_memray/tracking_api.cpp` and `src/memray/_memray/hooks.cpp` around the stack-trace frames, then use the coredump to identify why allocation tracking diverges; done means the pytest command no longer segfaults and remains consistent with core memray.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.