[ci-scan-feedback] KPI Tracker

Agentic workflow metadata (ci-failure-scan-feedback)

Workflow artifact: ci-scan-feedback
Artifact kind: kpi-tracker
Window start (first recorded scanner run): **2026-05-21**

Tracking quality of `[ci-scan]` (detection) and `[ci-fix]` (mitigation) issues, PRs, and loop-in comments since 2026-05-21. Updated every tick of [ci-failure-scan-feedback.lock.yml](https://github.com/dotnet/runtime/blob/main/.github/workflows/ci-failure-scan-feedback.lock.yml). To raise a concern, comment here or on any `[ci-scan]`/`[ci-fix]` issue/PR; the next tick reads in-scope feedback and either opens a `[ci-scan-feedback]` PR with prompt edits or pushes to the existing one.

## Snapshot — 2026-09-14T13:00Z

_The carried-forward prompt edit is **emitted this tick**: the single-PR slot freed when #133150 (`[ci-scan-feedback] Allow fixer to read scanner-authored KBE bodies`) merged (2026-09-14), so this run opens a `[ci-scan-feedback]` PR tightening `create-kbe.instructions.md` variation 8 so crossgen2 / ILC / ReadyToRun compile-phase aborts key on **compiler + invariant exception + leg root** and treat the per-work-item / assembly name (`Interop`, `TypeMapApp`, `b27077`, `Regression_4`, `LoaderClassloader`) as volatile incidental context — the failure mode that drove the scanner wrong-closure rate to **31.6%** (🔴), where the same compile abort was re-filed under different work-item names and auto-closed as `duplicate` (verified this tick: #133489 `TypeMapApp` and #133563 `b27077`/`Regression_4` both closed `duplicate`, #133596 cDAC closed `completed`). **Metric caveat:** this tick ran with `gh` unauthenticated (credentials stripped after checkout) and the MCP `search_issues`/`search_pull_requests` integrity gate removed every bot-authored `[ci-scan]`/`[ci-fix]` artifact from results, so the 7d/30d cohort counts below **could not be recomputed** and are carried forward from the last verified tick (2026-09-13T12:56Z); direct `issue_read`/`pull_request_read` by number still works and confirmed the dedup family above. Integrity gate filtered **all** in-scope search reads this tick (≥20 per query)._

### Activity (last 7d)

| stream | opened | good | wrong/unmerged |
|---|---|---|---|
| Scanner KBE issues | 24 | 12 | 8 |
| Fixer `[ci-fix]` PRs (confident) | 0 | 0 | 0 |
| Fixer `[ci-fix]` PRs (help-wanted) | 0 | 0 | 0 |
| Fixer loop-in comments | 0 | 0 engaged | — |

"good" = KBEs closed `completed`, PRs merged, loop-ins with a maintainer reply. "wrong/unmerged" = KBEs closed `not_planned`/`duplicate`, PRs closed without merge (note: a help-wanted PR closed unmerged is expected, not necessarily a miss). _Counts carried forward from 2026-09-13; not recomputable this tick (see caveat above)._

### Scanner quality (last 30d)

| metric | count | rate |
|---|---|---|
| KBE closures | 57 | — |
| Wrong KBE closures | 18 | 31.6% |
| Maintainer rejection comments | 0 | — |
| Duplicate KBEs | 16 | — |

_Counts carried forward from 2026-09-13; not recomputable this tick. All 18 wrong closures were automated dedup with no maintainer pushback (0 rejection comments)._

### Fixer quality (last 30d)

| metric | count | rate |
|---|---|---|
| Fix PR closures | 1 | — |
| Fix PRs merged | 0 | — |
| Fix PRs rejected (maintainer pushback) | 0 | n/a (1<5) |
| Loop-in comments (posted / engaged) | 0 / 0 | — |
| Loop-in / help-wanted mis-attributions | 0 | — |

_Counts carried forward from 2026-09-13. The only 30d fixer closure (#131981, help-wanted) was closed by a MEMBER as a neutral duplicate of #131515 — not maintainer pushback, so not counted as a rejection._

### Outage signals (analyzed CI)

| signal | threshold | 24h | 7d | status |
|---|---|---|---|---|
| New-KBE burst | day > 2x trailing 30d median (min 3) | 5 / median 3 | peak day 5 | 🟢 |
| Build-break spike | >= 2 in any 24h | 1 | 8 | 🔴 |
| Multi-pipeline outage (distinct legs proxy) | >= 3 distinct legs in 24h | 5 distinct | 5 distinct (peak day) | 🔴 |
| KBE re-filed after maintainer close | any in 7d | 0 | 0 | 🟢 |
| Scanner wrong-closure rate (30d) | >= 30% with `scan_closed_30d >= 10` | — | 31.6% | 🔴 |
| Fixer rejection rate (30d) | >= 30% with `fix_closed_30d >= 5` | — | n/a | 🟢 |

_Outage-signal values carried forward from 2026-09-13; the 24h/7d windows were not recomputable this tick due to the integrity gate + unauthenticated `gh`._

```
details: Build-break spike — carried forward from 2026-09-13: 2026-09-10 had 3 [ci-scan] build breaks (crossgen2 TypeLoadException #133562/#133563, cDAC universal-DBI #133596); 2026-09-09 had 2 (crossgen2 TypeMapApp #133489, superpmi-replay Helix-API #133493)
details: Multi-pipeline outage — carried forward from 2026-09-13: 5 distinct legs in the latest 24h window (windows-x64 TestReadyToRun_Libraries #133778, windows-x86 NativeAOT_Libs #133779, cDAC/DAC Windows runtime-diagnostics #133810, coreclr linux x64 tracing interpreter #133811, osx-arm64 NoDynamicCodeCompiled interpreter #133812)
details: Scanner wrong-closure rate — carried forward from 2026-09-13: 18 of 57 KBE closures in 30d were duplicate/not_planned (16 duplicate + 2 not_planned), all automated dedup with no maintainer pushback. The crossgen2 R2R / TypeMap / NativeAOT-ILC compile-phase family is re-filed under different incidental per-work-item test/assembly names (verified this tick: #133489 TypeMapApp, #133563 b27077/Regression_4) and closed as dup. The [ci-scan-feedback] PR opened this tick tightens create-kbe.instructions.md variation 8 to key compile-phase aborts on compiler+invariant-exception+leg-root and treat the per-work-item name as volatile.
```> Generated by [CI Outer-Loop Failure Scanner — Feedback](https://github.com/dotnet/runtime/actions/runs/34846308036) · opus48 · 480.8 AIC · ⌖ 19.9 AIC · ⊞ 18.7K · [◷](https://github.com/search?q=repo%3Adotnet%2Fruntime+is%3Aissue+%22gh-aw-workflow-call-id%3A+dotnet%2Fruntime%2Fci-failure-scan-feedback%22&type=issues)