python / python/cpython

`groupby_next` data race on free-threaded builds

Abierto
#150,791 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

extension-modules topic-free-threading type-bug
Lenguaje dominante
Python
Estrellas
77.2k
Forks
35.9k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

Bug report

Bug description:

Two threads calling next on the same groupby object concurrently in a free-threaded build race on the currgrouper field, corrupting the iterator's internal state and producing AttributeError on slot accesses of live objects.

groupby_next has no Py_BEGIN_CRITICAL_SECTION guard. The first thing it does is write gbo->currgrouper = NULL:

https://github.com/python/cpython/blob/c5516e7e371f7b273eb37c7b65f14ef14ee81f11/Modules/itertoolsmodule.c#L531-L540

Later, after the loop exits, it calls _grouper_create, which writes parent->currgrouper = igo:

https://github.com/python/cpython/blob/c5516e7e371f7b273eb37c7b65f14ef14ee81f11/Modules/itertoolsmodule.c#L624-L637

If Thread A is past line 633 (just stored the new grouper pointer) while Thread B is at line 537 (about to store NULL), Thread B overwrites the pointer Thread A just wrote. Both are plain pointer stores with no synchronisation.


The following script reproduces the condition under which this happens.

import itertools, threading

class K:
    __slots__ = ("v",)
    def __init__(self, v): self.v = v
    def __eq__(self, o): return isinstance(o, K) and self.v == o.v
    def __hash__(self): return hash(self.v)

def consume(g):
    try:
        while True:
            _, _ = next(g)
    except StopIteration:
        pass

keys = [K(i) for i in range(500_000)]
g = itertools.groupby(keys)
threads = [threading.Thread(target=consume, args=(g,)) for _ in range(8)]
for t in threads: t.start()
for t in threads: t.join()

On a free-threaded build, this results in the following Python logs...

Exception in Thread-15 (consume):
  File "<python-input-2>", line 6, in __eq__
    def __eq__(self, o): return isinstance(o, K) and self.v == o.v
                                                     ^^^^^^
AttributeError: 'K' object has no attribute 'v'

... and the following TSan logs.

WARNING: ThreadSanitizer: data race (pid=20464)
  Write of size 8 at 0x0003026e4a88 by thread T2:
    #0 groupby_next itertoolsmodule.c:537 (python.exe:arm64+0x10042a8b8)
    #1 builtin_next bltinmodule.c:1770

  Previous write of size 8 at 0x0003026e4a88 by thread T1:
    #0 _grouper_create itertoolsmodule.c:633 (python.exe:arm64+0x10042ac5c)
    #1 groupby_next itertoolsmodule.c:570 (python.exe:arm64+0x10042ac5c)
    #2 builtin_next bltinmodule.c:1770
CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs
  • gh-150792

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

Comienza en Modules/itertoolsmodule.c, en groupby_next y _grouper_create, y luego ejecuta el reproducer concurrente proporcionado en una compilación free-threaded con ThreadSanitizer. Se considera terminado cuando el reproducer ya no informe de la race de currgrouper ni produzca el AttributeError descrito.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
c, python
Área
backend, testing-qa
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bien especificado
Aptitud para principiantes
25/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.