General issue Python extractor fails in macOS App Sandbox: `PermissionError: [Errno 1] Operation not permitted` from `_multiprocessing.SemLock`
- Ngôn ngữ chính
- CodeQL
- Star
- 10.1k
- Fork
- 2.1k
- Merge trung bình
- 2 ngày 15 giờ
- Pull request đã merge (30 ngày)
- 141
Mô tả
## Summary
The Python extractor unconditionally uses `multiprocessing.Queue` and `multiprocessing.Process`, which require POSIX semaphores (`sem_open()`). In macOS App Sandbox environments (Seatbelt), `ipc-posix-sem` is denied at the kernel level, making `codeql database create --language=python` impossible.
No existing issue covers this — searched for `SemLock`, `semaphore`, `PermissionError macos`, `sandbox`, `multiprocessing` with zero matches.
## Environment
- CodeQL CLI: 2.25.6 (Homebrew cask, Apple Silicon)
- Python extractor version: 7.1.8
- macOS: Darwin 24.6.0 (Sequoia, arm64)
- Python: 3.12 and 3.14 (both fail identically)
- Sandbox: macOS Seatbelt (`sandbox-exec`) — used by Claude Code, Codex, and other sandboxed developer tools
## Reproduction
Run `codeql database create` inside any macOS App Sandbox that denies `ipc-posix-sem`:
```bash
# Minimal test — verify semaphores are blocked in your environment:
python3 -c "import multiprocessing; multiprocessing.get_context('spawn').Queue()"
# PermissionError: [Errno 1] Operation not permitted
# Then:
echo 'print("hello")' > /tmp/test.py
codeql database create /tmp/codeql-db --language=python --source-root=/tmp --overwrite
```
Fails at:
```
File ".../python3src.zip/semmle/logging.py", line 85, in __init__
self.queue = ctx.Queue()
...
_multiprocessing.SemLock(kind, value, maxvalue, self._make_name(), unlink_now)
PermissionError: [Errno 1] Operation not permitted
```
If the logger is patched to bypass this, a second identical failure occurs in `semmle/worker.py:115` (`ExtractorPool.__init__` → `ctx.Queue(proc_count*2)`).
## Affected Code
1. **`python/tools/python3src.zip → semmle/logging.py:84`** — `Logger.__init__` unconditionally creates `multiprocessing.Queue()` and spawns a `Process` for log message routing, regardless of verbosity level.
2. **`python/tools/python3src.zip → semmle/worker.py:115-116`** — `ExtractorPool.__init__` creates `multiprocessing.Queue` and `multiprocessing.Process` workers for parallel extraction.
Both use `multiprocessing.get_context('spawn')` on macOS, which calls `sem_open()`.
## Why This Matters
macOS Seatbelt sandboxing is increasingly common in developer tooling — Claude Code, GitHub Codex CLI, Gemini CLI, and third-party sandbox wrappers all use it. The `ipc-posix-sem` denial is standard in these profiles. As AI-assisted development grows, more developers will hit this when running CodeQL from sandboxed terminals.
## Suggested Fix
Add a fallback to `threading.Thread` + `queue.Queue` when `multiprocessing` is unavailable or fails. This is the same pattern used for AWS Lambda (where `/dev/shm` is unavailable) and Docker containers with restricted IPC namespaces.
A minimal change: catch `PermissionError`/`OSError` in `Logger.__init__` and `ExtractorPool.__init__`, falling back to thread-based equivalents. Single-threaded extraction already works correctly (verified with a patched extractor scanning 132 Python files).
Hướng dẫn đóng góp
Hướng nghiên cứu
Start with python/tools/python3src.zip/semmle/logging.py:84 and semmle/worker.py:115-116, then reproduce the failure with the sandboxed multiprocessing Queue command and `codeql database create`. Trace both queue and process initialization paths, and verify that Python database creation completes when POSIX semaphores are denied, including the logger and extractor pool paths.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- macos, python
- Lĩnh vực
- devtools, operating-systems
- Loại issue
- Lỗi
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức phù hợp với người mới
- 52/100