github / github/codeql

General issue Python extractor fails in macOS App Sandbox: `PermissionError: [Errno 1] Operation not permitted` from `_multiprocessing.SemLock`

オープン
#21,956 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る
question
主要言語
CodeQL
スター
10.1k
フォーク
2.1k
平均マージ
2日 15時間
マージ済み PR(30日)
141

説明

## Summary

The Python extractor unconditionally uses `multiprocessing.Queue` and `multiprocessing.Process`, which require POSIX semaphores (`sem_open()`). In macOS App Sandbox environments (Seatbelt), `ipc-posix-sem` is denied at the kernel level, making `codeql database create --language=python` impossible.

No existing issue covers this — searched for `SemLock`, `semaphore`, `PermissionError macos`, `sandbox`, `multiprocessing` with zero matches.

## Environment

- CodeQL CLI: 2.25.6 (Homebrew cask, Apple Silicon)
- Python extractor version: 7.1.8
- macOS: Darwin 24.6.0 (Sequoia, arm64)
- Python: 3.12 and 3.14 (both fail identically)
- Sandbox: macOS Seatbelt (`sandbox-exec`) — used by Claude Code, Codex, and other sandboxed developer tools

## Reproduction

Run `codeql database create` inside any macOS App Sandbox that denies `ipc-posix-sem`:

```bash
# Minimal test — verify semaphores are blocked in your environment:
python3 -c "import multiprocessing; multiprocessing.get_context('spawn').Queue()"
# PermissionError: [Errno 1] Operation not permitted

# Then:
echo 'print("hello")' > /tmp/test.py
codeql database create /tmp/codeql-db --language=python --source-root=/tmp --overwrite
```

Fails at:
```
File ".../python3src.zip/semmle/logging.py", line 85, in __init__
self.queue = ctx.Queue()
...
_multiprocessing.SemLock(kind, value, maxvalue, self._make_name(), unlink_now)
PermissionError: [Errno 1] Operation not permitted
```

If the logger is patched to bypass this, a second identical failure occurs in `semmle/worker.py:115` (`ExtractorPool.__init__` → `ctx.Queue(proc_count*2)`).

## Affected Code

1. **`python/tools/python3src.zip → semmle/logging.py:84`** — `Logger.__init__` unconditionally creates `multiprocessing.Queue()` and spawns a `Process` for log message routing, regardless of verbosity level.

2. **`python/tools/python3src.zip → semmle/worker.py:115-116`** — `ExtractorPool.__init__` creates `multiprocessing.Queue` and `multiprocessing.Process` workers for parallel extraction.

Both use `multiprocessing.get_context('spawn')` on macOS, which calls `sem_open()`.

## Why This Matters

macOS Seatbelt sandboxing is increasingly common in developer tooling — Claude Code, GitHub Codex CLI, Gemini CLI, and third-party sandbox wrappers all use it. The `ipc-posix-sem` denial is standard in these profiles. As AI-assisted development grows, more developers will hit this when running CodeQL from sandboxed terminals.

## Suggested Fix

Add a fallback to `threading.Thread` + `queue.Queue` when `multiprocessing` is unavailable or fails. This is the same pattern used for AWS Lambda (where `/dev/shm` is unavailable) and Docker containers with restricted IPC namespaces.

A minimal change: catch `PermissionError`/`OSError` in `Logger.__init__` and `ExtractorPool.__init__`, falling back to thread-based equivalents. Single-threaded extraction already works correctly (verified with a patched extractor scanning 132 Python files).

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

python/tools/python3src.zip/semmle/logging.py:84 と semmle/worker.py:115-116 から始め、sandboxed multiprocessing Queue コマンドと `codeql database create` を使って失敗を再現します。Queue とプロセスの両方の初期化パスを追跡し、POSIX セマフォが拒否された場合でも、logger と extractor pool のパスを含めて Python データベースの作成が完了することを確認します。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
macos, python
領域
devtools, operating-systems
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
静か
明瞭さ
明確に書かれている
初心者へのやさしさ
52/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。