General issue Python extractor fails in macOS App Sandbox: `PermissionError: [Errno 1] Operation not permitted` from `_multiprocessing.SemLock`
- 主要言語
- CodeQL
- スター
- 10.1k
- フォーク
- 2.1k
- 平均マージ
- 2日 15時間
- マージ済み PR(30日)
- 141
説明
## Summary
The Python extractor unconditionally uses `multiprocessing.Queue` and `multiprocessing.Process`, which require POSIX semaphores (`sem_open()`). In macOS App Sandbox environments (Seatbelt), `ipc-posix-sem` is denied at the kernel level, making `codeql database create --language=python` impossible.
No existing issue covers this — searched for `SemLock`, `semaphore`, `PermissionError macos`, `sandbox`, `multiprocessing` with zero matches.
## Environment
- CodeQL CLI: 2.25.6 (Homebrew cask, Apple Silicon)
- Python extractor version: 7.1.8
- macOS: Darwin 24.6.0 (Sequoia, arm64)
- Python: 3.12 and 3.14 (both fail identically)
- Sandbox: macOS Seatbelt (`sandbox-exec`) — used by Claude Code, Codex, and other sandboxed developer tools
## Reproduction
Run `codeql database create` inside any macOS App Sandbox that denies `ipc-posix-sem`:
```bash
# Minimal test — verify semaphores are blocked in your environment:
python3 -c "import multiprocessing; multiprocessing.get_context('spawn').Queue()"
# PermissionError: [Errno 1] Operation not permitted
# Then:
echo 'print("hello")' > /tmp/test.py
codeql database create /tmp/codeql-db --language=python --source-root=/tmp --overwrite
```
Fails at:
```
File ".../python3src.zip/semmle/logging.py", line 85, in __init__
self.queue = ctx.Queue()
...
_multiprocessing.SemLock(kind, value, maxvalue, self._make_name(), unlink_now)
PermissionError: [Errno 1] Operation not permitted
```
If the logger is patched to bypass this, a second identical failure occurs in `semmle/worker.py:115` (`ExtractorPool.__init__` → `ctx.Queue(proc_count*2)`).
## Affected Code
1. **`python/tools/python3src.zip → semmle/logging.py:84`** — `Logger.__init__` unconditionally creates `multiprocessing.Queue()` and spawns a `Process` for log message routing, regardless of verbosity level.
2. **`python/tools/python3src.zip → semmle/worker.py:115-116`** — `ExtractorPool.__init__` creates `multiprocessing.Queue` and `multiprocessing.Process` workers for parallel extraction.
Both use `multiprocessing.get_context('spawn')` on macOS, which calls `sem_open()`.
## Why This Matters
macOS Seatbelt sandboxing is increasingly common in developer tooling — Claude Code, GitHub Codex CLI, Gemini CLI, and third-party sandbox wrappers all use it. The `ipc-posix-sem` denial is standard in these profiles. As AI-assisted development grows, more developers will hit this when running CodeQL from sandboxed terminals.
## Suggested Fix
Add a fallback to `threading.Thread` + `queue.Queue` when `multiprocessing` is unavailable or fails. This is the same pattern used for AWS Lambda (where `/dev/shm` is unavailable) and Docker containers with restricted IPC namespaces.
A minimal change: catch `PermissionError`/`OSError` in `Logger.__init__` and `ExtractorPool.__init__`, falling back to thread-based equivalents. Single-threaded extraction already works correctly (verified with a patched extractor scanning 132 Python files).
コントリビューションガイド
調査の方向性
python/tools/python3src.zip/semmle/logging.py:84 と semmle/worker.py:115-116 から始め、sandboxed multiprocessing Queue コマンドと `codeql database create` を使って失敗を再現します。Queue とプロセスの両方の初期化パスを追跡し、POSIX セマフォが拒否された場合でも、logger と extractor pool のパスを含めて Python データベースの作成が完了することを確認します。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- macos, python
- 領域
- devtools, operating-systems
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 静か
- 明瞭さ
- 明確に書かれている
- 初心者へのやさしさ
- 52/100