anthropics / anthropics/skills
skill-creator: run_eval.py's nested claude -p subprocess inherits parent stdin, causing an intermittent stall
- Ngôn ngữ chính
- Python
- Star
- 176k
- Fork
- 20.8k
- Merge trung bình
- 7 giờ 21 phút
- Pull request đã merge (30 ngày)
- 5
Mô tả
## Problem
`run_single_query()` in `skills/skill-creator/scripts/run_eval.py` spawns the nested eval `claude -p` via `subprocess.Popen(cmd, stdout=..., stderr=..., cwd=..., env=...)` without redirecting `stdin`. The child inherits the parent's stdin file descriptor.
When `run_eval.py`/`run_loop.py` itself runs nested inside a `claude` session (a supported, intended use — the file explicitly strips `CLAUDECODE` from `env` to allow this), the child process can block briefly waiting on/polling the inherited stdin before it gives up and proceeds, adding an observed ~3s stall per query. At `--num-workers` > 1 this multiplies across concurrent subprocesses and inflates wall-clock eval time without any corresponding signal.
## Repro
Run any eval from inside a `claude -p` or Claude Code session:
python -m scripts.run_loop --eval-set eval.json --skill-path --model
Observe each `run_single_query` call taking ~3s longer than the same call run from a plain (non-nested) shell.
## Fix
Single-line change — explicitly discard the child's stdin instead of inheriting it:
```diff
--- a/skills/skill-creator/scripts/run_eval.py
+++ b/skills/skill-creator/scripts/run_eval.py
@@ -84,6 +84,7 @@ def run_single_query(
process = subprocess.Popen(
cmd,
+ stdin=subprocess.DEVNULL, # nested claude -p must not wait on/inherit our stdin
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
cwd=project_root,
```
## Note for maintainers
Neither open PR #1323 nor #1298 (both reworking this same function for the 0%-recall bug) touches the Popen call — this fix is orthogonal and should be folded into whichever lands, or applied standalone if merged first.
---
*Disclosure: this issue was researched and written by Claude (Claude Code)*
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.