getsentry / getsentry/sentry-python

Segfault in contextvars HAMT during isolation_scope() / new_scope() with uvicorn multiprocessing workers

Abierto
#5,190 3 comentarios 0 reacciones 1 asignado Reclamado por @sl0thentr0py Ver en GitHub
Bug Errors Python
Lenguaje dominante
Python
Estrellas
2.2k
Forks
669
Merge medio
1 d 1 h
PR fusionados (30 d)
213

Descripción

### How do you use Sentry?

Self-hosted/on-premise

### Version

2.45.0

### Steps to Reproduce

**Environment**

- sentry-sdk version: 2.45.0
- Python version: 3.10.x
- OS: Linux (x86_64)
- Framework: FastAPI + uvicorn with --workers 10 (multiprocessing spawn)
- Event loop: uvloop 0.21.0

**Problem**

Workers crash with SIGSEGV when Sentry SDK calls _current_scope.set() inside isolation_scope() or new_scope(). The crash occurs in Python's HAMT (Hash Array Mapped Trie) implementation used by contextvars.

GDB analysis shows that the HAMT node pointer has been corrupted - it points to a PyFunctionObject (_safe_repr_wrapper from sentry_sdk/serializer.py) instead of a HAMT node. When Python tries to clone this "node", it reads the function's vectorcall field as if it were b_array, then crashes trying to Py_INCREF a C function pointer.

Stack trace:

```
#0 _Py_INCREF (op=0x7b4d9a872260 <_PyFunction_Vectorcall>) at ./Include/object.h:472
#1 _Py_XINCREF (op=0x7b4d9a872260 <_PyFunction_Vectorcall>) at ./Include/object.h:558
#2 hamt_node_bitmap_clone (node=0x7b4c301c7e20) at Python/hamt.c:583
#3 hamt_node_bitmap_assoc (...) at Python/hamt.c:771
#4 _PyHamt_Assoc (...) at Python/hamt.c:2308
#5 contextvar_set (var=0x7b4d9916a5c0, val=) at Python/context.c:738
#6 PyContextVar_Set (ovar=<_contextvars.ContextVar at remote 0x7b4d9916a5c0>, ...) at Python/context.c:285
...
#12 _PyEval_EvalFrame (...) for file sentry_sdk/scope.py, line 1785, in isolation_scope
```

Python stack trace

```
File "sentry_sdk/scope.py", line 1785, in isolation_scope
current_token = _current_scope.set(forked_current_scope)
File "contextlib.py", line 135, in __enter__
return next(self.gen)
File "sentry_sdk/integrations/asgi.py", line 198, in _run_app
with sentry_sdk.isolation_scope() as sentry_scope:
```

Corrupted HAMT node:

```
(gdb) p *node
$2 = {
ob_base = {ob_type = 0x7b4d9aa8a7a0 },
b_bitmap = 2587471808,
b_array = {'_safe_repr_wrapper'} # Does not look like HAMT data
}

(gdb) p node->b_array[0]
$5 = '_safe_repr_wrapper'

(gdb) x/20x 0x7b4d9a872260
0x7b4d9a872260 <_PyFunction_Vectorcall>: ....
```

Sentry configuration

```
sentry_sdk.init(
dsn=settings.SENTRY_DSN,
traces_sample_rate=1.0,
profiles_sample_rate=0.0,
enable_tracing=True,
send_default_pii=True,
integrations=[
FastApiIntegration(),
HttpxIntegration(),
AioHttpIntegration(),
RedisIntegration(),
],
)
```

How to reproduce

1. FastAPI app with sentry-sdk 2.45.0
2. Run with uvicorn app:app --workers 10 (multiprocessing spawn)
3. Send concurrent HTTP requests
4. Workers eventually crash with SIGSEGV

Observations
- Crash happens in spawned worker processes, not the main process
- The corrupted pointer always points to Sentry SDK's _safe_repr_wrapper function or _PyFunction_Vectorcall
- Workers that don't use Sentry SDK (different project, same infrastructure) don't crash
- Same codebase works fine in single-process mode

We tried setting OPENBLAS_NUM_THREADS=1 and it did not help.

It appears that a function object created during Sentry serialization (_safe_repr_wrapper) is somehow being written to a location that should contain a HAMT node pointer.

This could be:
- A use-after-free where HAMT memory is reused for function objects
- A buffer overflow in serialization code
- Race condition in scope/context handling with multiprocessing

### Expected Result

No segfault

### Actual Result

Eventual segfault

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.