python / python/cpython

Document that bf_getbuffer must prevent GC (only under freethreading?) to write to its `view` argument

Aperta
#130,409 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

docs topic-free-threading
Lingua principale
Python
Stelle
77.2k
Fork
35.9k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

https://docs.python.org/3/c-api/typeobj.html#c.PyBufferProcs.bf_getbuffer

does not mention any requirement for the callee to hold the GIL or otherwise prevent Python GC to write to the view argument. One might think this is safe, because one has no particular reason to think that the view object is shared between threads and that the bf_getbuffer implementation has exclusive access to it.

However, the view object can be allocated from GC memory, e.g., in https://github.com/python/cpython/blob/4f07fd59cbd2656ae31e164098b8d10c5b7e51f0/Objects/memoryobject.c#L72
and this means that we may see concurrent GC traversals if we do not prevent GC (e.g., by holding the GIL).

Under free-threading mode, I saw the following race in the JAX test suite which I believe stems from this problem:

WARNING: ThreadSanitizer: data race (pid=268192)
  Read of size 8 at 0x7fffc2221a38 by thread T69 (mutexes: read M0):
    #0 mbuf_traverse /__w/jax/jax/cpython/Objects/memoryobject.c:134:5 (python3.13+0x27f190) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)
    #1 update_refs /__w/jax/jax/cpython/Python/gc_free_threading.c:441:5 (python3.13+0x44854a) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)
    #2 _mi_heap_area_visit_blocks /__w/jax/jax/cpython/Objects/mimalloc/heap.c:630:14 (python3.13+0x2a9fe4) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)
    #3 mi_heap_area_visitor /__w/jax/jax/cpython/Objects/mimalloc/heap.c:681:12 (python3.13+0x2aa547) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)
    #4 mi_heap_visit_areas_page /__w/jax/jax/cpython/Objects/mimalloc/heap.c:661:10 (python3.13+0x2aa547)
    #5 mi_heap_visit_pages /__w/jax/jax/cpython/Objects/mimalloc/heap.c:46:12 (python3.13+0x2aa547)
    #6 mi_heap_visit_areas /__w/jax/jax/cpython/Objects/mimalloc/heap.c:667:10 (python3.13+0x2aa547)
    #7 mi_heap_visit_blocks /__w/jax/jax/cpython/Objects/mimalloc/heap.c:692:10 (python3.13+0x2aa547)
    #8 gc_visit_heaps_lock_held /__w/jax/jax/cpython/Python/gc_free_threading.c:267:14 (python3.13+0x4446ff) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)
    #9 gc_visit_heaps /__w/jax/jax/cpython/Python/gc_free_threading.c:306:11 (python3.13+0x4446ff)
    #10 deduce_unreachable_heap /__w/jax/jax/cpython/Python/gc_free_threading.c:614:5 (python3.13+0x4458db) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)
    #11 gc_collect_internal /__w/jax/jax/cpython/Python/gc_free_threading.c:1126:15 (python3.13+0x4458db)
    #12 gc_collect_main /__w/jax/jax/cpython/Python/gc_free_threading.c:1239:5 (python3.13+0x4458db)
    #13 _Py_RunGC /__w/jax/jax/cpython/Python/gc_free_threading.c:1684:5 (python3.13+0x44730e) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)

 Previous write of size 8 at 0x7fffc2221a38 by thread T68 (mutexes: read M0):
    #0 __tsan_memset  (python3.13+0xda21d) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)
    #1 memset /usr/include/x86_64-linux-gnu/bits/string_fortified.h:59:10 (xla_extension.so+0xa7a60ec) (BuildId: 0cb922163d5a5c8d99583339143c71c4c2e776b1)
    #2 xla::(anonymous namespace)::PyArray_bf_getbuffer(_object*, Py_buffer*, int)::$_0::operator()() const /proc/self/cwd/external/xla/xla/python/py_array.cc:1601:5 (xla_extension.so+0xa7a60ec)
    #3 xla::(anonymous namespace)::PyArray_bf_getbuffer(_object*, Py_buffer*, int) /proc/self/cwd/external/xla/xla/python/py_array.cc:1518:25 (xla_extension.so+0xa7a60ec)
    #4 PyObject_GetBuffer /__w/jax/jax/cpython/Objects/abstract.c:442:15 (python3.13+0x1ba90d) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)
    #5 _PyManagedBuffer_FromObject /__w/jax/jax/cpython/Objects/memoryobject.c:97:9 (python3.13+0x27fdf5) (BuildId: 5001c71441642d7e6efdfc82265e7ee63d55228e)

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia con il riferimento alla C API all’indirizzo docs.python.org/3/c-api/typeobj.html, concentrandoti su PyBufferProcs.bf_getbuffer, e per contesto esamina il sito di allocazione collegato in Objects/memoryobject.c. Chiarisci se il requisito si applica solo al free-threading mode e documenta come le implementazioni devono proteggere le scritture nell’argomento view; il lavoro sarà considerato completato quando la documentazione dell’API dichiarerà esplicitamente il requisito di sincronizzazione applicabile.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
documentation
Tipo di issue
Documentazione
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
45/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.