Optimize deep-review skill for token usage

Problem

The deep-review skill currently runs 5 Opus invocations per review — 1 orchestrator + 4 parallel subagents — all with extended thinking enabled (via ultrathink on the orchestrator, and Opus's default reasoning on subagents). Every subagent receives the full diff, meaning the diff tokens are duplicated ~5x. There is no visibility into how many tokens a review actually consumes, making it hard to measure the impact of optimizations.

Telemetry constraint

Claude Code does not expose per-subagent token usage from within a skill. The /cost command gives session-level totals only — no breakdown by Agent call. Hooks cannot intercept API usage data. The only path to per-agent granularity is the Agent SDK (query() returns total_cost_usd, usage, and per-model breakdowns), which operates outside the skill framework. This shapes the investigation approach below.

Investigation Plan

Phase 1: Session-level baseline (no code changes)

Run /deep-review against 3–5 branches of varying size (small fix, medium feature, large refactor). For each run:

1. Note /cost before starting the review.
2. Run the review.
3. Note /cost after the review completes.
4. Record the delta (total cost, total tokens) and the branch metadata (diff line count, number of files).

Repeat under different configurations by editing the skill prompt locally:

Configuration	What to change
Baseline (all-Opus, ultrathink)	No changes — current behavior
Tiered models	Set Test Coverage and Design & Simplification agents to model: "sonnet"
No subagent thinking	Add "Do not use extended thinking." to each subagent prompt
Both optimizations	Tiered models + no subagent thinking

This gives 12–20 data points (3–5 branches x 4 configs) with session-level cost deltas. No instrumentation code required.

Phase 2: Qualitative comparison

For each configuration pair, compare the review outputs side-by-side:

• Precision: Did the optimized config produce false positives the baseline didn't?
• Recall: Did the optimized config miss findings the baseline caught?
• Severity accuracy: Were findings mis-classified (e.g., a 🔴 demoted to 💡)?
• Coherence: Did the orchestrator's synthesis step degrade when subagent output quality dropped?

Score each optimized config as: no regression / minor regression / unacceptable regression.

Phase 3: Per-agent instrumentation (only if Phase 1 shows meaningful savings)

Build a small Agent SDK harness (TypeScript or Python script, not a skill) that:

1. Spawns the same 4 review agents via query() calls.
2. Collects per-agent total_cost_usd, input/output/thinking token counts, and wall-clock time.
3. Writes results to a JSON file for analysis.

This harness is a benchmarking tool, not a replacement for the skill. Use it to get the per-agent breakdown that Claude Code can't provide, and to validate that session-level deltas from Phase 1 are consistent with the sum of per-agent costs.

Phase 4: Apply optimizations

Based on data from Phases 1–3, update the skill prompt. Candidate changes:

Optimization	Where to change	Risk
Tiered models for subagents	model: "sonnet" on specific Agent calls in Step 2	Lower reasoning quality on those agents
Drop thinking from subagents	Add instruction to subagent prompts	Subagents may miss subtle issues
Targeted diff chunking	Send only relevant hunks per agent (e.g., test files to Test Coverage)	Agent loses cross-file context

Phase 5: Validate

Re-run Phase 1 and Phase 2 against the same branches to confirm cost reduction and no quality regression.

Evaluation targets

Agent	Current model	Candidate	Rationale
Correctness & Security	opus	opus	Likely needs deep reasoning — keep as-is
Test Coverage	opus	sonnet	Pattern matching against test files; less complex reasoning
Design & Simplification	opus	sonnet	Style/complexity checks may not need Opus
Assumptions & Contracts	opus	opus or sonnet	Needs to trace callers — evaluate quality tradeoff

Acceptance Criteria

• Session-level cost baseline documented for 3+ branch sizes across 4 configurations
• Qualitative comparison of review outputs across configurations
• Recommendation on which optimizations to adopt, with data
• If optimizations are adopted: skill prompt updated, before/after cost comparison documented
• Decision on whether per-agent SDK harness (Phase 3) is worth building, based on Phase 1 results

Notes

• The skill prompt is at plugins/deep-review/commands/deep-review.md
• Model is set per-agent via model: "opus" in the Agent tool call (Step 2, line 94)
• ultrathink keyword on line 10 triggers extended thinking for the orchestrator
• /cost is the only session-level telemetry available without code changes
• The Agent SDK's query() is the only path to per-agent token breakdowns, but requires building outside the skill framework
• Quality must not regress — a cheaper review that misses findings is worse than the current cost

Start with plugins/deep-review/commands/deep-review.md, then run /deep-review on branches of different sizes while recording /cost before and after each configuration. Compare cost and review quality across the four configurations; done means the baseline, qualitative results, recommendation, and any adopted prompt changes are documented, with a decision on whether an Agent SDK harness is warranted.