`pathlib.Path.mkdir()` raises `FileExistsError` for non-existent path due to TOCTOU race condition

Bug report

Bug description:

pathlib.Path.mkdir() raises FileExistsError for non-existent path due to TOCTOU race condition

Summary

pathlib.Path.mkdir(exist_ok=True) can incorrectly raise FileExistsError when a directory is deleted between the os.mkdir() call and the subsequent is_dir() check. This occurs because is_dir() returns False both when a path exists as a non-directory (the intended error case) and when the path doesn't exist at all (a TOCTOU race condition that should be handled differently).

Bug Details

Current problematic code

In Lib/pathlib/_local.py (Python 3.13+) or Lib/pathlib.py (earlier versions):

try:
    os.mkdir(self, mode)
except FileNotFoundError:
    if not parents or self.parent == self:
        raise
    self.parent.mkdir(parents=True, exist_ok=True)
    self.mkdir(mode, parents=False, exist_ok=exist_ok)
except OSError:
    # Cannot rely on checking for EEXIST, since the operating system
    # could give priority to other errors like EACCES or EROFS
    if not exist_ok or not self.is_dir():
        raise

The logic if not exist_ok or not self.is_dir(): raise assumes is_dir() returning False means "path exists but is not a directory." However, is_dir() also returns False when the path doesn't exist at all, creating a TOCTOU vulnerability.

Race condition sequence

1. Thread/Process A calls path.mkdir(exist_ok=True)
2. os.mkdir() raises FileExistsError (directory existed at that moment)
3. Thread/Process B deletes the directory before A's next line executes
4. A calls self.is_dir() → returns False (path no longer exists)
5. Code incorrectly re-raises FileExistsError for a non-existent path

Expected vs. Actual Behavior

Expected

When mkdir(exist_ok=True) is called:

• If path exists as a directory: return successfully (no error)
• If path exists as a non-directory: raise FileExistsError
• If path doesn't exist: retry mkdir or raise FileNotFoundError
• NEVER raise FileExistsError for a path that doesn't exist

Actual

FileExistsError can be raised even when the path doesn't exist if it's deleted during the race window.

Reproduction

Minimal example (monkey-patched simulation)

#!/usr/bin/env python3
"""Minimal reproduction of pathlib.mkdir() TOCTOU race condition."""

from pathlib import Path
import tempfile

# Create a test directory
tmpdir = Path(tempfile.mkdtemp())
test_path = tmpdir / "race_dir"
test_path.mkdir()

# Monkey-patch is_dir() to delete the directory before checking
original_is_dir = Path.is_dir

def racing_is_dir(self):
    if self == test_path and self.exists():
        self.rmdir()  # Simulate race: delete during is_dir() call
    return original_is_dir(self)

Path.is_dir = racing_is_dir

# This will raise FileExistsError even though path doesn't exist!
try:
    test_path.mkdir(exist_ok=True)
    print("No error - race didn't trigger")
except FileExistsError as e:
    print(f"FileExistsError: {e}")
    print(f"Path exists: {test_path.exists()}")  # False!
    print("✓ Bug reproduced: FileExistsError raised but path doesn't exist")

# Cleanup
Path.is_dir = original_is_dir
tmpdir.rmdir()

Output:

FileExistsError: [Errno 17] File exists: '/tmp/tmpXXXXXX/race_dir'
Path exists: False
✓ Bug reproduced: FileExistsError raised but path doesn't exist

Real-world scenario

This occurs naturally in multiprocessing/multithreading scenarios with:

• Thread A: Creating shared memory arrays with Path.mkdir(parents=True, exist_ok=True)
• Thread B: Cleaning up arrays via __del__, removing empty parent directories with rmdir()

The race occurs when cleanup happens between mkdir's FileExistsError and the is_dir() check.

Related Issues

• BPO-29694 / GH-73880: Fixed race in mkdir(parents=True) but didn't address deletion races
• BPO-35192: Duplicate of 29694, users still hitting races on older Python versions
• GH-139001: New pathlib race in Python 3.14t free-threading

Workaround

Until fixed, applications must implement retry logic:

def robust_mkdir(path, max_retries=3):
    for attempt in range(max_retries):
        try:
            path.mkdir(parents=True, exist_ok=True)
            return
        except FileExistsError:
            if path.is_dir():
                return  # Race resolved
            if attempt == max_retries - 1:
                raise
            time.sleep(0.01 * (2 ** attempt))

Linked PRs

• gh-143162

Start in Lib/pathlib/_local.py for Python 3.13+ (or Lib/pathlib.py for earlier versions), focusing on Path.mkdir() and its is_dir() check. Use the monkey-patched reproduction to observe the race behavior. Done means mkdir(exist_ok=True) does not raise FileExistsError when the path has been deleted, while preserving the documented behavior for directories and non-directories.