skill: /mutation-check — verify the tests for a change actually fail without it

Problem

standards/CLAUDE.md already says "Tests that pass regardless of whether the feature works are worthless". Nothing operationalizes it, so it stays an aspiration.

On TimZander/AudioClassifier #195 I hand-rolled the same loop three times — back up the file, apply a perl -0pi -e mutation, run the suite, count failures, restore, verify — and got it wrong twice:

1. A stale pattern silently applied no mutation. The regex still targeted a code shape that had been edited away, so sed matched nothing, the suite passed, and my script reported SURVIVED — GAP. A false gap, indistinguishable from real uncovered behaviour.
2. A restore wiped real work. The script restored from a backup a subagent had captured earlier, silently reverting ~15 lines. git status looked clean afterwards.

When it worked, it was decisive and repeatedly changed decisions:

• 7 of 8 mutants surviving condemned a suite that looked like coverage — including deletion of the very line the PR added.
• 13 of 13 caught validated the rewrite.
• A survivor exposed a test that claimed to pin a guard it could never reach (ok: false short-circuited before the check).
• It disproved my own "13 of 13" claim on a later pass by testing a mutant I hadn't thought of.

Why a skill rather than documentation

The loop is mechanical, error-prone, and stateful in a dangerous way — it edits tracked source and must restore it. Those properties argue for scripted tooling over a prose rule:

• Restore must be guaranteed, via trap ... EXIT, not left to a happy path. My hand-rolled version lost work exactly here.
• Mutation patterns must be re-derived from the current diff. Hand-written patterns go stale the moment the code is edited — which is guaranteed, since you edit code in response to what mutation testing tells you.
• "Not applied" must be distinguishable from "survived." Conflating them produces false gaps that send you chasing coverage you already have. This is the single highest-value thing a script gets right and a human gets wrong.

A prose rule can say "revert the fix and check the test fails". It cannot prevent any of the three failures above.

Proposed behaviour

/mutation-check [path...] [--base <branch>]

1. Refuse to run on a dirty working tree — the whole thing hinges on a trustworthy restore.
2. Derive candidate mutations from the branch diff vs base: revert changed arguments to their base value, invert conditionals, delete added calls, empty added collections, drop added guards.
3. For each mutant: apply → assert it actually changed the file → run the relevant suite → record caught (with which tests) / survived / not-applied.
4. Restore unconditionally via trap ... EXIT; verify against git diff and fail loudly if the tree does not match.
5. Report a table. Call out survivors as uncovered behaviour, list not-applied separately as a tool defect, not a finding, and state plainly that the mutant set bounds the claim.

Acceptance criteria

• Refuses to run on a dirty working tree
• Derives mutants from the diff vs base rather than hardcoded patterns
• Guarantees restore on success, failure, and interrupt (trap ... EXIT), then verifies it against git diff
• Distinguishes "survived" from "mutation not applied" and reports them separately
• Reports which tests caught each mutant
• Output states that results are bounded by the mutant set generated

Notes

Pairs with the standards rule proposed in the test-validity issue (assert the outcome, not the call; mutation-verify bug-fix tests) — that issue defines the practice, this makes it cheap enough to actually do.

Prior art worth checking before building: Stryker (JS/.NET), mutmut/cosmic-ray (Python). A thin wrapper over an existing mutator may beat a bespoke implementation — but the diff-scoped mutant generation and the not-applied detection are the parts that mattered here, and general-purpose mutators are usually whole-file and slow.

Read standards/CLAUDE.md and inspect the proposed /mutation-check entry point, then compare Stryker, mutmut, and cosmic-ray before choosing an approach. Done means the command meets every acceptance criterion: safe dirty-tree handling and restoration, diff-scoped mutations, separate caught/survived/not-applied results, caught-test reporting, and bounded-result wording.