getsentry / getsentry/sentry-python

Memory leak: DedupeIntegration pins exception tracebacks in asyncio apps

Abierto
#6,094 7 comentarios 1 reacción 1 asignado Reclamado por @ericapisani Ver en GitHub
Bug Python Waiting for: Product Owner
Lenguaje dominante
Python
Estrellas
2.2k
Forks
669
Merge medio
1 d 1 h
PR fusionados (30 d)
213

Descripción

### How do you use Sentry?

Self-hosted/on-premise

### Version

2.56.0 (Python 3.12.6)

### Steps to Reproduce

1. `sentry_sdk.init` with only `LoggingIntegration` + `DedupeIntegration` enabled.
2. In a long-lived asyncio task: build a local variable of ~1 MB so the frame carries that
much data, raise `ValueError`, log with `logger.exception(...)`, keep the task running.
3. The exception, its traceback, and the 1 MB local stay alive for the lifetime of the task (and maybe after its death)

### Expected Result

After `logger.exception(...)` returns and the `except` block exits, the exception, its
traceback, and all its frame locals become unreachable and memory are freed

### Actual Result

The exception stays alive for the whole lifetime of the task. Memory grows with every new
captured exception on a new task, and the 1 MB local (plus the whole traceback chain)
stays pinned until the task itself ends.

RAM usage growth:
Image

## What I found

`DedupeIntegration._last_seen` is a `ContextVar` that stores the last captured exception.
The SDK tries to store it as `weakref.ref(exc)`, but Python builtins (`ValueError`,
`TypeError`, `KeyError`, ...) do not support weakref. On `TypeError` the SDK falls back to
a strong reference

Under asyncio every long-lived task keeps its own `ContextVar` state, so the strong
reference lives as long as the task. Through that reference the SDK pins:
- the exception,
- `exc.__traceback__`,
- every frame local in the traceback.

In our web crawler frame locals held response payloads in the 500 KB-1 MB range, which is where
the RSS growth came from

## Fix (what I did)

Replace the dedupe processor with one that stores `weakref.ref(exc)` when possible and
silently skips on `TypeError`. Dedupe is lost for builtins, leak is gone

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.