anthropics / anthropics/skills

skill-creator: run_eval.py's nested claude -p subprocess inherits parent stdin, causing an intermittent stall

Đang mở
#1,414 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
176k
Fork
20.8k
Merge trung bình
7 giờ 21 phút
Pull request đã merge (30 ngày)
5

Mô tả

## Problem

`run_single_query()` in `skills/skill-creator/scripts/run_eval.py` spawns the nested eval `claude -p` via `subprocess.Popen(cmd, stdout=..., stderr=..., cwd=..., env=...)` without redirecting `stdin`. The child inherits the parent's stdin file descriptor.

When `run_eval.py`/`run_loop.py` itself runs nested inside a `claude` session (a supported, intended use — the file explicitly strips `CLAUDECODE` from `env` to allow this), the child process can block briefly waiting on/polling the inherited stdin before it gives up and proceeds, adding an observed ~3s stall per query. At `--num-workers` > 1 this multiplies across concurrent subprocesses and inflates wall-clock eval time without any corresponding signal.

## Repro

Run any eval from inside a `claude -p` or Claude Code session:

python -m scripts.run_loop --eval-set eval.json --skill-path --model

Observe each `run_single_query` call taking ~3s longer than the same call run from a plain (non-nested) shell.

## Fix

Single-line change — explicitly discard the child's stdin instead of inheriting it:

```diff
--- a/skills/skill-creator/scripts/run_eval.py
+++ b/skills/skill-creator/scripts/run_eval.py
@@ -84,6 +84,7 @@ def run_single_query(

process = subprocess.Popen(
cmd,
+ stdin=subprocess.DEVNULL, # nested claude -p must not wait on/inherit our stdin
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
cwd=project_root,
```

## Note for maintainers

Neither open PR #1323 nor #1298 (both reworking this same function for the 0%-recall bug) touches the Popen call — this fix is orthogonal and should be folded into whichever lands, or applied standalone if merged first.

---
*Disclosure: this issue was researched and written by Claude (Claude Code)*

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.