`run_eval.py` always reports 0% recall: it probes a `.claude/commands/` slash-command proxy instead of the real installed skill

**Plugin:** `skill-creator` (this repo, `plugins/skill-creator/.../scripts/run_eval.py`)
**Claude Code:** 2.1.185
**Affects:** `run_eval.py` and therefore `run_loop.py` (the description-optimization loop)

## Summary

The description-optimization eval always reports **recall = 0% / precision = 100% / accuracy = 50%** for every skill and every description — i.e. the target skill is *never* detected as triggering. This makes the entire `run_loop` description optimizer unusable: it can never observe a should-trigger query firing, so it has no real signal to optimize against.

This has been reported before in anthropics/claude-code#36570 and anthropics/claude-code#32184, where it was attributed to *"`claude -p` can't auto-trigger skills in headless mode"* and closed **"not planned."** **That attribution is now incomplete/outdated** — see below. The remaining, still-present root cause lives in this plugin's harness.

## Root cause

`run_eval.py` does not test the real installed skill. For each query it:

1. Writes a throwaway **slash command** to `/.claude/commands/.md` (comment in source: *"Creates a command file in .claude/commands/ so it appears in Claude's available_skills list"*), then
2. runs `claude -p --output-format stream-json` and counts a "trigger" only if the model's first tool call is a `Skill` (or `Read`) tool whose argument contains that **temp command's** name.

Two problems make this structurally unable to detect a trigger:

- **Commands are not model-auto-invoked the way skills are.** A file in `.claude/commands/` is a user-typed slash command (`/name`), not an entry the model selects from `available_skills` and invokes via the `Skill` tool. So the probe artifact never fires.
- **Even when the model *does* auto-trigger a skill, the name won't match.** The model triggers the real installed skill (its own `name`), but the detector is looking for the temp command's randomized `clean_name`. During `run_loop`, the description being optimized is the temp command's — the real skill's description is never varied — so optimization can't affect triggering either.

## Evidence that this is the harness, not `claude -p`

On 2.1.185, `claude -p` **does** auto-trigger real installed skills. Minimal check:

```bash
# real skill installed at ~/.claude/skills//SKILL.md
env -u CLAUDECODE claude -p "" \
--output-format stream-json --include-partial-messages --verbose \
--model --max-turns 1 | grep -o '"skill":"[^"]*"'
# -> "skill":"" # i.e. it triggered
```

1. Install a skill at `~/.claude/skills//SKILL.md` with a clear description.
2. Create an eval set of should-trigger queries.
3. Run `python -m scripts.run_eval --eval-set .json --skill-path --model ` → recall 0% for all should-trigger queries, despite the skill triggering fine when `claude -p` is run directly.

Start with plugins/skill-creator/.../scripts/run_eval.py and trace how it creates the temporary .claude/commands/ proxy, invokes claude -p, and identifies triggers; then inspect the affected run_loop.py flow. Reproduce the reported eval command against an installed skill and compare its result with the real Skill invocation described in the issue. Done means the eval varies and detects the actual installed skill and produces meaningful recall and specificity results.

Written by the indexing model from the issue text.