python / python/cpython

nogil inconsistent empty list while sorting

Open
#126,559 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

3.13 3.14 interpreter-core topic-free-threading type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

Hi,

We're a research group focused on testing concurrent runtimes. Our work-in-progress prototype found a violation of atomicity on the current nogil build when using concurrent operations on the same list. The program below shows the wrong behavior.

from threading import Thread,Barrier,Lock

def t1(b1,l,r):
    b1.wait()
    r[0] = l.__len__()

def t2(b1,l,r):
    b1.wait()
    r[0] = l.__str__()

def t3(b1,l):
    b1.wait()
    l.sort()

def normalSetTest(i):

    sharedList = [1,2,3] * 100
    barrier = Barrier(2)
    res = [ None ]
    threads = [
            Thread(target= t1, args=(barrier,sharedList,res,)),
            #Thread(target= t2, args=(barrier,sharedList,res,)),
            Thread(target= t3, args=(barrier,sharedList,)),
            ]


    for t in threads:
        t.start()
    for t in threads:
        t.join()

    if res[0] in [ 0, '[]' ]:
        print("\tfound bug: " + str(res[0]))

print("test begin...")

for n in range(0,10):
    threads = []

    for i in range(0,1):
        threads.append(Thread(target= normalSetTest, args=(n,)))

    for t in threads:
        t.start()

    for t in threads:
        t.join()

print("test Done")

A list with 300 integers is sorted in t3 while another thread either gets the size of the list t1 or turns the list into a string t2. Running the code above will show threads t1 and t2 finding an inconsistent empty list with length 0 or turned into the string "[]".
Our tool did not find any other interesting values: either it's an expected value (original list or sorted list) OR it's empty.

Sample output:

test begin...
        found bug: 0
        found bug: 0
        found bug: 0
        found bug: 0
        found bug: 0
test Done

We're happy to provide more details about this bug, and to help developers reproducing it.

Output of python -VV: Python 3.14.0a1+ experimental free-threading build (heads/main:faa3272fb8d, Oct 29 2024, 09:14:25) [GCC 14.2.1 20240805]

@flypoodles and @overlorde are part of the team, adding them so they get notified about further discussion.

I believe this issue is part of the ongoing conversation on #126136 about acceptable behaviors of containers operated by many threads concurrently, described in the original nogil PEP: https://peps.python.org/pep-0703/#container-thread-safety

CPython versions tested on:

3.13, 3.14, CPython main branch

Operating systems tested on:

Linux

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied reproducer on a free-threading CPython build, then read the PEP 703 container thread-safety section and the discussion in issue #126136. The expected behavior for concurrent list sorting, length, and string conversion must be established before determining whether a fix and regression test are required.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.