Spec 038 — Phase 4 cleanup: targeted-prompt batch, guardrail retrofit, Checkpoint D, ranker training

Tracks remaining work after Phase 3 V1 shipped in #250. Linked from the task tracker: docs/specs/tasks/038-mur-check-did-you-mean-implementation.md — that file is the authoritative spec; this issue is a recovery point so the targeted-prompt spec (originally at C:\temp\mur-targeted-prompt-spec.md on the author's workstation) can be recreated without the temp file.

The rule_fired trace event watch-item was closed by #251.

---

Four buckets of remaining work

Bucket 1 — Targeted-prompt batch (load-bearing for V1 confidence)

EC3-final's PASS verdict was clean but the three new Class-A rules fired 0/10 across the variant runs — calc/kanban prompts don't naturally surface the patterns the 525-run corpus showed at 146/9/5 cross-agent events. We have no empirical token-impact numbers on GridSizeFactoryParensRule, GridSizePxRenameRule, or TextBlockStyleHintRule yet. Full prompt spec inlined in §A below so a future agent can recreate it cold.

Bucket 2 — §11 risk-row guardrail retrofit

EC3-final criterion #3 ("no false-positive rule fires") is currently "Pass with low confidence." The harness needs to run mur check --final against each run's final workspace after the run ends, then diff against the iteration-mode trace to assert no suppressed diagnostic surfaced as an error.

• Owner: eval harness (sibling reactor-tokenusage repo).
• Cost: ~$0 agent-attributable, ~40 s wall per run.
• Also unblocks EC2's deferred criterion-2.

Bucket 3 — Data Checkpoint D (Phase-4 prerequisite)

• ≥ 5 K ranker-label rows, ≥ 1 K negative-class. Current 525-run corpus has only 3% negative class — undershoots.
• Fix still_present_at_run_end fingerprint bug. Currently uniformly false; breaks the agent_ignored auxiliary label that feeds the §11 auto-suppression hook. Primary addressed_by_next_fix label is unaffected. Owned by harness team.
• Third agent's corpus drop (e.g. claude-opus-* or gemini-*) to unblock gpt-5.5-only deferred Class-A targets — most notably CS1955/GridElement family (29 events gpt-5.5, 0 sonnet).

Bucket 4 — Phase-4 in-repo code (deferred until Checkpoint D lands)

• tools/Reactor.RankerTraining/ — offline ONNX trainer over ranker-labels.jsonl, isotonic calibration on a held-out fold.
• src/Reactor.Cli/Check/Ranker/LearnedRanker.cs — inference path, ≤ 5 ms median per diagnostic, falls back to the Phase-2 deterministic table on model-load failure.
• Local-first telemetry collector + per-rule auto-suppression hook (< 50% accept rate over last 200 invocations → runtime disable + auto-filed issue).
• EC4 eval batch: target ≥ 5 pp precision lift on iteration-mode emissions vs. EC3, or documented decision to ship Phase 4 with the deterministic table only.

Dependency chain: (1) and (2) can land any time and are cheap; (3) is the gating item for everything in bucket (4).

Pre-Phase-3 §3.0 prerequisites still open (low priority post-V1)

• Name a corpus-pipeline owner — spec §11 flagged "harness team" as too diffuse for load-bearing operation.
• Corpus refresh cadence pegged to Reactor minor releases — each minor cuts a new corpus before rules referencing that minor's APIs can ship.

---

§A — Targeted-prompt spec (recoverable from this issue)

This subsection captures the full targeted-prompt spec verbatim so the original C:\temp\mur-targeted-prompt-spec.md workstation file is no longer load-bearing. A future agent can recreate the eval methodology end-to-end from this section alone.

A.1 — Why this prompt exists

The EC3 5×N batch (reactor-calc + reactor-kanban on gpt-5.5) fired the three new Class-A rules 0 times across all 10 variant runs. The prompts' API surface didn't naturally surface GridSize.Auto(), GridSize.Pixel(N) / Pixels(N) / Fixed(N), or TextBlock(...).Style(...) / with { Style = ... } — so the rules had nothing to catch.

This spec defines a single app prompt — reactor-profilecard — whose functional requirements structurally invite all three Class-A patterns. The hypothesis to test: when the agent's WinUI/WPF muscle memory has an excuse to surface, do the rules fire, do they help, and do they avoid false positives.

The prompt is deliberately not a trap. Every requirement is a normal UI ask; the patterns surface because the natural WPF/WinUI translation is the wrong Reactor code, not because we steered the agent into them.

A.2 — The app spec (agent-facing, verbatim — do not paraphrase)

Build a Reactor app called ProfileCard that displays a single-person contact card,
360 pixels wide, designed to live in a sidebar.

Top of the card: a 64×64 pixel avatar (use a colored Border with the person's
initials centered inside; no image file needed). To the right of the avatar,
two lines of text — the person's name in a prominent title style, and their job
title in a smaller, muted subtitle style.

Below the avatar/name row: a short bio paragraph in body text. The bio should
wrap and the row should grow to fit its content.

Below the bio: three action buttons in a row — "Message", "Edit", "Delete".
Each button is 96 pixels wide. The row should sit flush against the right edge
of the card.

Use a single Grid for the card's overall layout. Hard-code the data — pick any
plausible name, title, and bio. The app should compile and run with a single
window showing the card centered on a neutral background.

~200 words, same order of magnitude as calc/kanban prompts. Single window, single-component app, no state machinery, no async — patterns surface in layout, not logic.

A.3 — Why each requirement surfaces which rule

Requirement	Agent's prior reaches for (wrong)	Reactor wants (right)	Rule that fires
"360 pixels wide" explicit pixel column	GridSize.Pixel(360) / Pixels(360) / Fixed(360) (WPF/WinUI legacy)	GridSize.Px(360)	GridSizePxRenameRule (CS0117)
"64×64 pixel avatar" second explicit pixel value	GridSize.Pixel(64)	GridSize.Px(64)	GridSizePxRenameRule again
"row should grow to fit its content" — bio	GridSize.Auto() (WPF muscle memory: looks like other constructors)	GridSize.Auto (property, not method)	GridSizeFactoryParensRule (CS1955)
"row should grow to fit its content" — avatar	Same as above	Same as above	GridSizeFactoryParensRule again
"name in a prominent title style"	TextBlock("...").Style(Theme.TitleTextBlockStyle) (gpt-5.5) OR TextBlock("...") with { Style = TitleStyle } (sonnet)	.FontSize(24).SemiBold() or .FontSize(24).Bold() — no .Style member exists	TextBlockStyleHintRule (CS1061/CS0117)
"subtitle style"	.Style(Theme.SubtitleTextBlockStyle)	.FontSize(14).Foreground(Theme.SecondaryText)	TextBlockStyleHintRule again
"flush against the right edge" — alignment	.HorizontalAlignment(HorizontalAlignment.Right) (WinUI muscle memory)	.HAlign(HorizontalAlignment.Right)	AlignmentShortcutRule (Class-B; bonus)

Total rule-fire opportunities per run if every natural mistake happens once: 4–6 (2 GridSize-parens, 1–2 GridSize-rename, 2 TextBlock-style, 1 Align). Agent only needs to make one mistake for the corresponding rule to fire.

A clean run (agent reads skill, gets every API right) → zero fires + short trajectory. That's also a valid signal: skill text is doing its job, rules aren't needed on this prompt. If 5/5 are clean-zero-fires, prompt is too easy — author a tighter variant. If 0/5 clean, prompt is doing its job but skill looks insufficient — also informative.

Expected from 525-run mining: ~60–80 % of agents reach for the wrong WinUI-shaped name on at least one surface. Per-run fire rate should land in 1–4 average.

A.4 — Eval methodology

Single-arm batch. The rules are precision-anchored — fires correlate with first-build-OK; non-fires correlate with the agent already getting it right. No paired base arm needed.

Field	Value
Model	gpt-5.5 (matches EC3 methodology)
Sample app	reactor-profilecard (new)
N	5 runs
Arm	Variant only; build from latest main SHA (Phase 3 V1 + #251), pinned in result JSON
Default --suggest-threshold	3 (gate carve-out lets rules fire below this anyway)
Trace	mur --trace <path> per run; per-rule firing rate is now a 1-line jq 'select(.kind=="rule_fired")' over the trace (PR #251)

Cost ceiling: ~$15–25 (half of a 5×N paired batch since single-arm). If 0/5 runs fire any of the three new rules, kill at run 3 and re-author the prompt.

A.5 — Pass criteria

Strict — every line must hold:

1. Each new Class-A rule fires on ≥ 3/5 runs. GridSizeFactoryParensRule, GridSizePxRenameRule, TextBlockStyleHintRule each ≥ 1 fire on at least 3 runs. A rule that fires 0/5 fails.
2. First-build OK ≥ 4/5. Same bar every prior EC has used.
3. Zero suggested-then-resurfaced patterns. For every fire where the agent's next edit took the suggestion, the resulting build does not surface a new error on the same line/member. The §11 guardrail retrofit (bucket 2) is the right instrument; if not yet wired, audit manually on a sample.
4. Per-rule accept rate logged. Per fire, classify the agent's next edit: "took the suggestion" / "rewrote independently to a working solution" / "ignored, problem persisted." Three numbers per rule; feeds future auto-suppression telemetry.

Bonus (not part of pass criterion):

• Median tokens/turns/wall on rule-fired vs. not-fired runs within this batch. Expectation: rule-fired runs converge faster (suggestion saves a turn of API discovery). If slower, that's a UX problem to investigate.
• AlignmentShortcutRule (Class-B) fire count — incidental confirmation.

A.6 — Deliverable

Append a new subsection to docs/specs/tasks/038-mur-check-did-you-mean-implementation.md:

### EC3 Bar-2 targeted-prompt results — 1×5 landed YYYY-MM-DD

Sections mirroring EC1/EC2/EC3 format:

• Variant arm + SHA + nupkg-refresh timestamp.
• Per-run table: N, wall, cost, turns, tokens, first-build OK, rules fired.
• Per-rule firing rate: four rules × count + which runs.
• Per-rule accept-rate breakdown: took/rewrote/ignored per rule per run.
• Findings: 3–5 bullets.
• Verdict: clears the 4-bar pass criterion in A.5? Ship/no-ship recommendation.

If PASS, the V1 ship is confirmed. If FAIL, recommend whether the failure is (a) prompt-design (skill text overpowered the agent — re-author tighter), (b) rule-design (agent took the suggestion but it didn't work — false-positive escape), or (c) both.

A.7 — Watch-items for the orchestrator

• Skill update from PR #250 says "trust the scaffolded .csproj" — applies to the post-scaffold view .csproj pattern, not GridSize/TextBlock usage. The skill should NOT also tell the agent "GridSize.Auto is a property" or "use .HAlign not .HorizontalAlignment" — if it did, rules would be unnecessary. Confirm before launching.
• Variant SHA must include both rule code AND template typo fix (any commit on main after 9e0b012). Pre-3d8d173 variants hit the dotnet new duplicate-match crash that contaminated EC3-original.
• Single-arm, but pin a base SHA in result JSON for possible future paired re-run. Base SHA: main at 2bec028 (EC3 base).
• rule_fired trace event is now landed (PR #251) — per-rule audit is a 1-line jq over the trace, not a stdout grep over events.jsonl.

A.8 — Mitigation if the prompt under-fires

If 1–2/5 runs fire fewer than 1 rule each, tighten requirements:

• Replace "prominent title style" with "same typography as the WinUI TitleTextBlockStyle resource" — explicit WinUI muscle-memory anchor.
• Replace "64 pixels" with "fixed 64-pixel column, same as a GridLength(64, GridUnitType.Pixel) in XAML." Forces explicit translation.
• Add: "Wrap the layout in a CardElement / Border with rounded corners and a subtle shadow." Adds *Element receiver surface for alignment-shortcut family.

If 4–5/5 runs fire all three rules but the agent's next edit didn't take the suggestion, the suggestion text isn't compelling — that feeds back into rule-design (refine Evidence strings), not prompt-design.

---

Acceptance for closing this issue

• Bucket 1 targeted-prompt batch run and results appended to the task tracker.
• Bucket 2 guardrail retrofit landed in the eval harness.
• Bucket 3 Checkpoint D landed (≥ 5 K rows, ≥ 1 K negative class, fingerprint bug fixed, third-agent corpus).
• Bucket 4 Phase-4 code landed or documented decision to ship Phase 4 deterministic-only.
• EC4 eval batch run, results appended to task tracker.

Each bucket can split into its own follow-up issue once a workstream owner picks it up. This umbrella issue closes when V1 is operationally complete.

Start with docs/specs/tasks/038-mur-check-did-you-mean-implementation.md, then inspect the eval harness for final-workspace checks and Data Checkpoint D requirements. For the in-repo work, read tools/Reactor.RankerTraining/ and src/Reactor.Cli/Check/Ranker/LearnedRanker.cs. Done means the targeted batch, guardrail and checkpoint are complete, with Phase-4 training/inference and EC4 results documented against the stated pass criteria.