IronLanguages / IronLanguages/ironpython3

IronPython shows weak memory behaviors not visible in CPython

Open
#1,972 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
2.8k
Forks
316
Avg merge
1d 9h
Merged PRs (30d)
1

Description

Hi,

We're a research group focused on testing concurrent runtimes. Our work-in-progress prototype found a behavior that happens in IronPython but not in CPython 3.14 (with or without the GIL) when using concurrent reads and writes on shared variables by different threads. The main idea is to have two threads T1 and T2 and two shared variables x and y such that:

#T1
write(x,1)
read(y)

#T2
write(y,1)
read(x)

There is no scheduling that explains both threads reading 0. However, we found this behavior is possible on IronPython on x86 and ARM64 architectures with the following program (please allow 20 minutes at least to observe):

from threading import Thread, Barrier, Lock
​
iters = 10000000
​
lock = Lock()
done = False
​
x = 0
y = 0
rx = -1
ry = -1
​
# write x, read y
def t0(b1, b2):
    global x
    global y
    global rx
    global ry
​
    while True:
        b1.wait()
​
        # TEST BEGIN
        x = 1
        tmp = y
        # TEST END
​
        with lock:
            ry = tmp
​
            if done:
                return
​
        b2.wait()
​
# write y, read x
def t1(b1, b2):
    global x
    global y
    global rx
    global ry
​
    while True:
        b1.wait()
​
        # TEST BEGIN
        y = 1
        tmp = x
        # TEST END
​
        with lock:
            rx = tmp
​
            if done:
                return
​
        b2.wait()
​
# 3, to synchronize t0, t1, and main thread
b1 = Barrier(3)
b2 = Barrier(3)
​
tt1 = Thread(target=t0, args=(b1, b2))
tt2 = Thread(target=t1, args=(b1, b2))
​
tt1.start()
tt2.start()
​
ret = 0
​
# repeat test "iters" amount of times
for i in range(iters):
    b1.wait()
    b2.wait()
​
    # by program order, at least one of rx or ry should hold the value "1"
    if ry == 0 and rx == 0:
        # weak behavior found, end program and clean up
        print(f"WEAK BEHAVIOR DETECTED ON ITERATION {i}")
        with lock:
            done = True
        b1.wait()
        tt1.join()
        tt2.join()
        ret = 1
​
        exit(ret)
​
​
    # reset values
    x = 0
    y = 0
    rx = -1
    ry = -1
​
print("Done")
print(f"Did not observe any weak behaviors")
​
with lock:
    done = True
​
b1.wait()
​
tt1.join()
tt2.join()
​
exit(ret)

We also observed the following test failing in x86 and ARM64 architectures, where IronPython allows the result y==2 && tmp==0, which cannot be explained by any interleaving of the operations:

#T1
write(x,1)
write(y, 1)

#T2
write(y,2)
tmp = read(x)

We also observed the following test failing in ARM64 architectures, where IronPython allows Thread T2 to observe the result y==1 && x==0, which cannot be explained by any interleaving of the operations:

#T1
write(x,1)
write(y,1)

#T2
read(y)
read(x)

All the behaviors described above are impossible in CPython 2.X, and CPython 3.X (with and without the GIL) in all architectures.

@mqbal is part of the team, adding them so they get notified about further discussion.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or existing test is named. Start by running the provided concurrent reproducer on x86 and ARM64 IronPython, then trace the threading and runtime memory-ordering paths involved. Done means the reported outcomes are no longer observable and regression coverage exists for the described cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, python
Domain
backend, operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.