deep-review: add a refute-pass phase, and assign severity after it rather than before

Problem

deep-review currently spawns parallel analysis agents (Step 2), reconciles their findings, and assigns severity in the output step. Nothing challenges a finding once it exists. The result is severity inflation, and it is measurable.

In one session I produced a review carrying eight 🔴 critical findings. Four agents were then pointed at those findings and told to refute them, defaulting to REFUTED where evidence was weak or the trigger contrived. One of the eight survived.

• Two were refuted outright. One alleged a symlink TOCTOU as a security defect; the refuter pointed out the tool already executes arbitrary argv from the same spec file, so the "exploit" was a longer, noisier path to what one bash -c does directly — no trust boundary is crossed. Another alleged a broken Python floor; the floor was already broken elsewhere in the repo, CI pinned a newer version, and the measurement behind the claim used the wrong instrument.
• Five were real but overgraded — genuine defects with contrived triggers, full recoverability, or behaviour matching what established tools in the same space already do.
• One was upheld and turned out to be worse than originally reported.

The bias had a consistent direction: severity inflated where the finding made the review look more necessary. That is the same bias the existing citation rule names for factual claims. Severity grades are claims too, and nothing currently checks them.

Why this belongs in the skill rather than in documentation

The same session had already added written working agreements to a repo's CLAUDE.md, and the very next PRs violated them. Prose in an already-read file does not fire when someone is moving quickly. A phase in the skill fires because it is in the path.

It is also mechanically repetitive — pair the findings, write the refute instruction, spawn the agents, reconcile, re-grade. That is exactly the shape that belongs in a skill.

Proposed change

A new phase between the existing parallel-analysis step and the output step:

1. Every 🔴, plus a sample of 🟡, goes to a refuter agent.
2. Refuters are instructed to default to REFUTED when evidence is weak, when the trigger requires contrived conditions, or when the behaviour matches an accepted convention in comparable tools. They are asked to construct the strongest argument for the current code before concluding, and to state whether the assigned severity is right, too high, or too low.
3. Refuters must reproduce by running, and must mark each conclusion as measured or inferred.
4. Severity is assigned after this pass, not before.
5. The output notes how many findings were refuted or downgraded — a review that reports its own correction rate is more trustworthy than one that does not.

Pair the claims (two per refuter) to bound cost, and give each refuter isolated scratch space — a mutating agent must never share a working tree with anything else measuring in it.

Acceptance criteria

• deep-review gains a refute phase that runs after analysis and before output
• Refuter prompts default to REFUTED and explicitly ask for the counter-argument
• Refuters are told to reproduce by running, and to mark measured vs inferred
• Severity is assigned only after the refute pass
• The output reports how many findings were downgraded or refuted
• Each refuter gets isolated scratch space; none share a working tree with another agent
• Cost is bounded — findings paired, not one agent per finding

Out of scope

Refuting 💡 suggestions. The cost is not justified and the downside of a wrong suggestion is low.

Start at the existing deep-review Step 2 analysis phase and the output step, tracing how findings are spawned, reconciled, and assigned severity. Implement the intervening refute phase using the listed acceptance criteria, then verify that refuted or downgraded counts, measured-versus-inferred conclusions, paired findings, and isolated scratch spaces are represented in the output.