General issue Python extractor fails in macOS App Sandbox: `PermissionError: [Errno 1] Operation not permitted` from `_multiprocessing.SemLock`
- 主要語言
- CodeQL
- 星號
- 10.1k
- 分支
- 2.1k
- 平均合併
- 2 天 15 小時
- 30 天內合併 PR
- 141
描述
## Summary
The Python extractor unconditionally uses `multiprocessing.Queue` and `multiprocessing.Process`, which require POSIX semaphores (`sem_open()`). In macOS App Sandbox environments (Seatbelt), `ipc-posix-sem` is denied at the kernel level, making `codeql database create --language=python` impossible.
No existing issue covers this — searched for `SemLock`, `semaphore`, `PermissionError macos`, `sandbox`, `multiprocessing` with zero matches.
## Environment
- CodeQL CLI: 2.25.6 (Homebrew cask, Apple Silicon)
- Python extractor version: 7.1.8
- macOS: Darwin 24.6.0 (Sequoia, arm64)
- Python: 3.12 and 3.14 (both fail identically)
- Sandbox: macOS Seatbelt (`sandbox-exec`) — used by Claude Code, Codex, and other sandboxed developer tools
## Reproduction
Run `codeql database create` inside any macOS App Sandbox that denies `ipc-posix-sem`:
```bash
# Minimal test — verify semaphores are blocked in your environment:
python3 -c "import multiprocessing; multiprocessing.get_context('spawn').Queue()"
# PermissionError: [Errno 1] Operation not permitted
# Then:
echo 'print("hello")' > /tmp/test.py
codeql database create /tmp/codeql-db --language=python --source-root=/tmp --overwrite
```
Fails at:
```
File ".../python3src.zip/semmle/logging.py", line 85, in __init__
self.queue = ctx.Queue()
...
_multiprocessing.SemLock(kind, value, maxvalue, self._make_name(), unlink_now)
PermissionError: [Errno 1] Operation not permitted
```
If the logger is patched to bypass this, a second identical failure occurs in `semmle/worker.py:115` (`ExtractorPool.__init__` → `ctx.Queue(proc_count*2)`).
## Affected Code
1. **`python/tools/python3src.zip → semmle/logging.py:84`** — `Logger.__init__` unconditionally creates `multiprocessing.Queue()` and spawns a `Process` for log message routing, regardless of verbosity level.
2. **`python/tools/python3src.zip → semmle/worker.py:115-116`** — `ExtractorPool.__init__` creates `multiprocessing.Queue` and `multiprocessing.Process` workers for parallel extraction.
Both use `multiprocessing.get_context('spawn')` on macOS, which calls `sem_open()`.
## Why This Matters
macOS Seatbelt sandboxing is increasingly common in developer tooling — Claude Code, GitHub Codex CLI, Gemini CLI, and third-party sandbox wrappers all use it. The `ipc-posix-sem` denial is standard in these profiles. As AI-assisted development grows, more developers will hit this when running CodeQL from sandboxed terminals.
## Suggested Fix
Add a fallback to `threading.Thread` + `queue.Queue` when `multiprocessing` is unavailable or fails. This is the same pattern used for AWS Lambda (where `/dev/shm` is unavailable) and Docker containers with restricted IPC namespaces.
A minimal change: catch `PermissionError`/`OSError` in `Logger.__init__` and `ExtractorPool.__init__`, falling back to thread-based equivalents. Single-threaded extraction already works correctly (verified with a patched extractor scanning 132 Python files).
貢獻指南
研究方向
Start with python/tools/python3src.zip/semmle/logging.py:84 and semmle/worker.py:115-116, then reproduce the failure with the sandboxed multiprocessing Queue command and `codeql database create`. Trace both queue and process initialization paths, and verify that Python database creation completes when POSIX semaphores are denied, including the logger and extractor pool paths.
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- macos, python
- 領域
- devtools, operating-systems
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 冷清
- 描述清晰度
- 描述清楚
- 新手友好度
- 52/100