anthropics / anthropics/skills

skill-creator: run_eval.py's nested claude -p subprocess inherits parent stdin, causing an intermittent stall

オープン 初心者向け
#1,414 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
176k
フォーク
20.9k
平均マージ
7時間 21分
マージ済み PR(30日)
5

説明

## Problem

`run_single_query()` in `skills/skill-creator/scripts/run_eval.py` spawns the nested eval `claude -p` via `subprocess.Popen(cmd, stdout=..., stderr=..., cwd=..., env=...)` without redirecting `stdin`. The child inherits the parent's stdin file descriptor.

When `run_eval.py`/`run_loop.py` itself runs nested inside a `claude` session (a supported, intended use — the file explicitly strips `CLAUDECODE` from `env` to allow this), the child process can block briefly waiting on/polling the inherited stdin before it gives up and proceeds, adding an observed ~3s stall per query. At `--num-workers` > 1 this multiplies across concurrent subprocesses and inflates wall-clock eval time without any corresponding signal.

## Repro

Run any eval from inside a `claude -p` or Claude Code session:

python -m scripts.run_loop --eval-set eval.json --skill-path --model

Observe each `run_single_query` call taking ~3s longer than the same call run from a plain (non-nested) shell.

## Fix

Single-line change — explicitly discard the child's stdin instead of inheriting it:

```diff
--- a/skills/skill-creator/scripts/run_eval.py
+++ b/skills/skill-creator/scripts/run_eval.py
@@ -84,6 +84,7 @@ def run_single_query(

process = subprocess.Popen(
cmd,
+ stdin=subprocess.DEVNULL, # nested claude -p must not wait on/inherit our stdin
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
cwd=project_root,
```

## Note for maintainers

Neither open PR #1323 nor #1298 (both reworking this same function for the 0%-recall bug) touches the Popen call — this fix is orthogonal and should be folded into whichever lands, or applied standalone if merged first.

---
*Disclosure: this issue was researched and written by Claude (Claude Code)*

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

Start in skills/skill-creator/scripts/run_eval.py at run_single_query(), where the issue points to the subprocess.Popen call for the nested claude -p command. Reproduce with python -m scripts.run_loop --eval-set eval.json --skill-path --model from inside a claude -p or Claude Code session. Done means the child process no longer inherits stdin and the nested eval no longer adds the observed per-query stall.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
cli, tooling
issue の種類
バグ
難易度
1/5
見積もり時間
1時間未満
活発さ
静か
明瞭さ
明確に書かれている
初心者へのやさしさ
72/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。