General issue Python extractor fails in macOS App Sandbox: `PermissionError: [Errno 1] Operation not permitted` from `_multiprocessing.SemLock`

Commencez par python/tools/python3src.zip/semmle/logging.py:84 et semmle/worker.py:115-116, puis reproduisez l’échec avec la commande sandboxed multiprocessing Queue et codeql database create. Suivez les chemins d’initialisation de Queue et des processus, puis vérifiez que la création de la base de données Python se termine lorsque les sémaphores POSIX sont refusés, y compris les chemins de logger et d’extractor pool.

Summary

The Python extractor unconditionally uses multiprocessing.Queue and multiprocessing.Process, which require POSIX semaphores (sem_open()). In macOS App Sandbox environments (Seatbelt), ipc-posix-sem is denied at the kernel level, making codeql database create --language=python impossible.

No existing issue covers this — searched for SemLock, semaphore, PermissionError macos, sandbox, multiprocessing with zero matches.

Environment

• CodeQL CLI: 2.25.6 (Homebrew cask, Apple Silicon)
• Python extractor version: 7.1.8
• macOS: Darwin 24.6.0 (Sequoia, arm64)
• Python: 3.12 and 3.14 (both fail identically)
• Sandbox: macOS Seatbelt (sandbox-exec) — used by Claude Code, Codex, and other sandboxed developer tools

Reproduction

Run codeql database create inside any macOS App Sandbox that denies ipc-posix-sem:

# Minimal test — verify semaphores are blocked in your environment:
python3 -c "import multiprocessing; multiprocessing.get_context('spawn').Queue()"
# PermissionError: [Errno 1] Operation not permitted

# Then:
echo 'print("hello")' > /tmp/test.py
codeql database create /tmp/codeql-db --language=python --source-root=/tmp --overwrite

Fails at:

File ".../python3src.zip/semmle/logging.py", line 85, in __init__
    self.queue = ctx.Queue()
...
_multiprocessing.SemLock(kind, value, maxvalue, self._make_name(), unlink_now)
PermissionError: [Errno 1] Operation not permitted

If the logger is patched to bypass this, a second identical failure occurs in semmle/worker.py:115 (ExtractorPool.__init__ → ctx.Queue(proc_count*2)).

Affected Code

1. python/tools/python3src.zip → semmle/logging.py:84 — Logger.__init__ unconditionally creates multiprocessing.Queue() and spawns a Process for log message routing, regardless of verbosity level.

2. python/tools/python3src.zip → semmle/worker.py:115-116 — ExtractorPool.__init__ creates multiprocessing.Queue and multiprocessing.Process workers for parallel extraction.

Both use multiprocessing.get_context('spawn') on macOS, which calls sem_open().

Why This Matters

macOS Seatbelt sandboxing is increasingly common in developer tooling — Claude Code, GitHub Codex CLI, Gemini CLI, and third-party sandbox wrappers all use it. The ipc-posix-sem denial is standard in these profiles. As AI-assisted development grows, more developers will hit this when running CodeQL from sandboxed terminals.

Suggested Fix

Add a fallback to threading.Thread + queue.Queue when multiprocessing is unavailable or fails. This is the same pattern used for AWS Lambda (where /dev/shm is unavailable) and Docker containers with restricted IPC namespaces.

A minimal change: catch PermissionError/OSError in Logger.__init__ and ExtractorPool.__init__, falling back to thread-based equivalents. Single-threaded extraction already works correctly (verified with a patched extractor scanning 132 Python files).