pytest-dev / pytest-dev/pyfakefs

`FakeOsModule.use_original` is not thread-safe under `asyncio.to_thread` workloads

Open
#1,317 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
750
Forks
99
Avg merge
19h 6m
Merged PRs (30d)
6

Description

[This bug was found, isolated, fixed, and reported by Claude Opus 4.7, with some minor edits by me.]

Describe the bug

FakeOsModule.use_original is a process-global class attribute flipped by the use_original_os() context manager. When one thread enters the context manager while another thread is mid-faked-filesystem-operation, the second thread observes use_original=True set by the first thread and dispatches to the real os module, raising a real PermissionError against paths that only exist in the fake filesystem.

Expected: every faked filesystem call inside Patcher stays faked, regardless of what other threads are doing.

Observed: intermittent PermissionError: [Errno 13] Permission denied: '<fake-path>' from real os.mkdir while a worker thread is calling Path.mkdir() against the fake filesystem.

Representative stack trace (from the plain repro below on 6.2.0):

Traceback (most recent call last):
  File ".../threading.py", line 1073, in _bootstrap_inner
    self.run()
  File ".../threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File ".../concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File ".../repro.py", line 6, in _write
    p.parent.mkdir(parents=True, exist_ok=True)
  File ".../pathlib/_local.py", line 770, in mkdir
    os.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '/fake-env'

The trace shows pathlib.Path.mkdiros.mkdir(self, mode) reaching the real kernel's mkdir, which has no knowledge of /fake-env (the path exists only inside pyfakefs's in-memory tree).

How To Reproduce

Plain repro:

import asyncio
from pathlib import Path
from pyfakefs.fake_filesystem_unittest import Patcher

def _write(p):
    p.parent.mkdir(parents=True, exist_ok=True)
    with p.open("w") as cf:
        cf.write("x")

async def _dispatch(p):
    await asyncio.to_thread(_write, p)

async def many():
    await asyncio.gather(*[
        _dispatch(Path(f"/fake-env/a/m{i}/deep/file")) for i in range(8)
    ])

with Patcher() as p:
    p.fs.create_dir("/fake-env")
    asyncio.run(many())

The repro's failure rate depends on whether another pyfakefs-internal code path (linecache during traceback formatting, RealPath wrappers in fake_pathlib, etc.) concurrently enters use_original_os() while the workers run.

Deterministic repro: fail reliably by adding hammer threads entering the context manager in a tight loop from within the Patcher context so they do not contend with pyfakefs's cold init:

import asyncio
import threading
from pathlib import Path
from pyfakefs.fake_filesystem_unittest import Patcher
from pyfakefs.fake_os import use_original_os

def _write(p):
    p.parent.mkdir(parents=True, exist_ok=True)
    with p.open("w") as cf:
        cf.write("x")

async def _dispatch(p):
    await asyncio.to_thread(_write, p)

async def many(root, n):
    await asyncio.gather(
        *[_dispatch(Path(f"{root}/a/m{i}/deep/file")) for i in range(n)]
    )

with Patcher() as p:
    stop = threading.Event()
    def hammer():
        while not stop.is_set():
            with use_original_os():
                pass
    hammers = [threading.Thread(target=hammer, daemon=True) for _ in range(4)]
    for h in hammers:
        h.start()
    try:
        for round_idx in range(3):
            root = f"/fake-env-{round_idx}"
            p.fs.create_dir(root)
            asyncio.run(many(root, 32))
    finally:
        stop.set()
        for h in hammers:
            h.join(timeout=1)

With hammers running inside the Patcher context, the failure is reliable.

Your environment

Linux-6.17.0-20-generic-x86_64-with-glibc2.39
Python 3.12.3 (main, Mar  3 2026, 12:15:18) [GCC 13.3.0]
pyfakefs 6.2.0
pytest 9.0.2

Also reproduces on current main (6.3.dev0, commit b907b2b).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading FakeOsModule.use_original and the use_original_os context manager in fake_os, then inspect the fake_pathlib paths involved in the repro. Run the deterministic asyncio.to_thread reproduction with hammer threads; done means concurrent faked filesystem calls never dispatch to the real os module and the repro completes without PermissionError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.