python / python/cpython

GC - Possible to indefinitely prevent collection of generation-1 objects by calling gc.collect(0) often

Abierto
#122,567 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

performance type-bug
Lenguaje dominante
Python
Estrellas
77.2k
Forks
35.9k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

Bug report

Bug description:

The example below will die from memory exhaustion, despite nearly all objects being eligible for collection and despite garbage collection not being disabled. See inline comments for an explanation of what I think is happening:

import gc
from typing import Any

class Temp:
  def __init__(self):
    # Cyclic reference to prevent deletion by ref count dropping to 0.
    self._x = self
    # Allocate a bunch of memory so that the process will die if
    # collection doesn't happen.
    self.memory = bytearray(1024*1024*512)

# Set some low thresholds so that collection should occur often.
gc.set_threshold(100, 3, 3)

while True:
  ref = Temp()
  # Promote the new Temp() object to generation-1. I think this is also resetting
  # the GCs view of how many objects were allocated at the "last collection". If
  # that's true, it will effectively stop the GC from ever being invoked
  # automatically, even though we're accumulating more and more objects,
  # because `threshold0` will never be exceeded.
  gc.collect(0)
  del ref
  # Temp() is now in generation-1 and eligible for collection, but gc.collect(1)
  # is never triggered, because there are no automatically invoked GCs at all.
  # Here we log the number of generation-1 objects, which continues to increase
  # until the process dies.
  print("Number of generation 1 objects: ", len(gc.get_objects(generation=1)))

Commenting out only the gc.collect(0) allows the Temp objects to be collected, and the process to run indefinitely. The fact that removing an explicit GC actually enables more GC to happen seems broken.

If what I think is happening is what's happening, then perhaps manually invoked GCs should not reset the GC's view on how many objects were allocated at the "last collection", since that would be what's effectively disabling the automatically triggered GCs (and hence the GCs at higher generations) from running.

CPython versions tested on:

3.11

Operating systems tested on:

macOS

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

No se nombra ningún archivo fuente ni ninguna prueba. Empieza ejecutando el reproductor proporcionado en CPython 3.11 y rastrea el comportamiento de gc.collect(0), los umbrales de la recolección automática y el crecimiento de la generación 1. El trabajo estará terminado cuando se confirme la interacción indicada y, si se verifica el comportamiento, se añadan una prueba de regresión adecuada y una corrección.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python
Área
backend
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
35/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.