pytest-dev / pytest-dev/pyfakefs
`FakeOsModule.use_original` is not thread-safe under `asyncio.to_thread` workloads
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 750
- Forks
- 99
- Avg merge
- 19h 6m
- Merged PRs (30d)
- 6
Description
[This bug was found, isolated, fixed, and reported by Claude Opus 4.7, with some minor edits by me.]
Describe the bug
FakeOsModule.use_original is a process-global class attribute flipped by the use_original_os() context manager. When one thread enters the context manager while another thread is mid-faked-filesystem-operation, the second thread observes use_original=True set by the first thread and dispatches to the real os module, raising a real PermissionError against paths that only exist in the fake filesystem.
Expected: every faked filesystem call inside Patcher stays faked, regardless of what other threads are doing.
Observed: intermittent PermissionError: [Errno 13] Permission denied: '<fake-path>' from real os.mkdir while a worker thread is calling Path.mkdir() against the fake filesystem.
Representative stack trace (from the plain repro below on 6.2.0):
Traceback (most recent call last):
File ".../threading.py", line 1073, in _bootstrap_inner
self.run()
File ".../threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File ".../concurrent/futures/thread.py", line 59, in run
result = self.fn(*self.args, **self.kwargs)
File ".../repro.py", line 6, in _write
p.parent.mkdir(parents=True, exist_ok=True)
File ".../pathlib/_local.py", line 770, in mkdir
os.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '/fake-env'
The trace shows pathlib.Path.mkdir → os.mkdir(self, mode) reaching the real kernel's mkdir, which has no knowledge of /fake-env (the path exists only inside pyfakefs's in-memory tree).
How To Reproduce
Plain repro:
import asyncio
from pathlib import Path
from pyfakefs.fake_filesystem_unittest import Patcher
def _write(p):
p.parent.mkdir(parents=True, exist_ok=True)
with p.open("w") as cf:
cf.write("x")
async def _dispatch(p):
await asyncio.to_thread(_write, p)
async def many():
await asyncio.gather(*[
_dispatch(Path(f"/fake-env/a/m{i}/deep/file")) for i in range(8)
])
with Patcher() as p:
p.fs.create_dir("/fake-env")
asyncio.run(many())
The repro's failure rate depends on whether another pyfakefs-internal code path (linecache during traceback formatting, RealPath wrappers in fake_pathlib, etc.) concurrently enters use_original_os() while the workers run.
Deterministic repro: fail reliably by adding hammer threads entering the context manager in a tight loop from within the Patcher context so they do not contend with pyfakefs's cold init:
import asyncio
import threading
from pathlib import Path
from pyfakefs.fake_filesystem_unittest import Patcher
from pyfakefs.fake_os import use_original_os
def _write(p):
p.parent.mkdir(parents=True, exist_ok=True)
with p.open("w") as cf:
cf.write("x")
async def _dispatch(p):
await asyncio.to_thread(_write, p)
async def many(root, n):
await asyncio.gather(
*[_dispatch(Path(f"{root}/a/m{i}/deep/file")) for i in range(n)]
)
with Patcher() as p:
stop = threading.Event()
def hammer():
while not stop.is_set():
with use_original_os():
pass
hammers = [threading.Thread(target=hammer, daemon=True) for _ in range(4)]
for h in hammers:
h.start()
try:
for round_idx in range(3):
root = f"/fake-env-{round_idx}"
p.fs.create_dir(root)
asyncio.run(many(root, 32))
finally:
stop.set()
for h in hammers:
h.join(timeout=1)
With hammers running inside the Patcher context, the failure is reliable.
Your environment
Linux-6.17.0-20-generic-x86_64-with-glibc2.39
Python 3.12.3 (main, Mar 3 2026, 12:15:18) [GCC 13.3.0]
pyfakefs 6.2.0
pytest 9.0.2
Also reproduces on current main (6.3.dev0, commit b907b2b).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading FakeOsModule.use_original and the use_original_os context manager in fake_os, then inspect the fake_pathlib paths involved in the repro. Run the deterministic asyncio.to_thread reproduction with hammer threads; done means concurrent faked filesystem calls never dispatch to the real os module and the repro completes without PermissionError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100