Impact Efficiency Report - 2026-06-17

## Executive Summary

Over the 180-day analysis window (2025-12-19 – 2026-06-17), agentic workflows consumed **43,055 AI Credits** while producing 7 in-scope outcomes (PRs with linked closing issues). Three accepted outcomes — two critical bug fixes and one improvement — delivered a **Total Outcome Value of 205**, yielding an **Impact Efficiency of 0.00476 value units/AIC**. The low ratio reflects that 993 PRs were excluded for lacking linked closing issues and that the largest AIC consumers (reporting, analysis workflows) produced no measurable PR outcomes. Direct workflow attribution is unavailable this cycle due to missing attribution fields in the PR dataset.

---

## Summary

| Metric | Value |
|---|---:|
| PRs analyzed with linked issue | merged: 3, closed: 4, open excluded: 0 |
| PRs excluded (no linked closing issue) | merged: 497, closed: 496 — total: **993** |
| Safe output outcomes | 0 |
| Outcomes mapped to objectives | 7 |
| Unmapped outcomes | 0 |
| Accepted outcome count | 3 |
| Total outcome value | **205** |
| AI Credits | **43,055.27** |
| Impact Efficiency | **0.00476** value units / AIC |

> AIC source: `aic-by-workflow.json` from 48 daily token-audit memory snapshots (primary; no live fallback).

---

## Agentic Work by Objective

All outcomes are unattributed to workflows, so per-category AIC and IE cannot be computed.

| Category | Map Value | Accepted | Outcome Value | AIC | IE | Examples |
|---|---:|---:|---:|---:|---:|---|
| **bug** | 85 | 2 | **170** | N/A | N/A | [PR #37863](https://github.com/github/gh-aw/pull/37863), [PR #37565](https://github.com/github/gh-aw/pull/37565) |
| **improvement** | 35 | 1 | **35** | N/A | N/A | [PR #38185](https://github.com/github/gh-aw/pull/38185) |

Root issues: [#37835](https://github.com/github/gh-aw/issues/37835) & [#37545](https://github.com/github/gh-aw/issues/37545) (`bug`,`compiler`,`workflows`); [#38143](https://github.com/github/gh-aw/issues/38143) (`improvement`,`quick-win`).

Four rejected (closed-without-merge) PRs were mapped to **improvement** (obj=35, outcome value=0): [#36767](https://github.com/github/gh-aw/pull/36767), [#36371](https://github.com/github/gh-aw/pull/36371), [#35906](https://github.com/github/gh-aw/pull/35906), [#35330](https://github.com/github/gh-aw/pull/35330) — all `[docs]` automation PRs.

**Highest AIC consumer:** `daily-experiment-report` (3,625 AIC, 2 runs) — no accepted outcomes.

---

## Which Workflows Drove That Impact

Workflow attribution for all 7 analyzed outcomes is **unavailable** — the PR datasets contain no `workflow_run_id`/`workflow_name` fields and `workflow-logs.json` covers only 10 runs from today.

**Unattributed bucket:** 7 outcomes, 3 accepted, Total Outcome Value = **205**

Top AIC-consuming workflows (none with attributed outcome value):

| Workflow | AIC | Runs |
|---|---:|---:|
| daily-experiment-report | 3,625 | 2 |
| Impact Efficiency Report | 2,647 | 3 |
| Contribution Check | 2,278 | 5 |
| Daily Semgrep Scan | 2,209 | 2 |
| Daily Firewall Logs Collector | 2,144 | 1 |
| Glossary Maintainer | 2,039 | 2 |

> ⚠️ No workflow was credited with delivered outcome value this cycle — attribution data was missing, not absent agentic output.

---

## Top Outcomes by Outcome Value

| Outcome | Workflow | Type | Root / Objective | Obj Value | Outcome Value |
|---|---|---|---|---:|---:|
| [PR #37863](https://github.com/github/gh-aw/pull/37863) — Fix push_to_pull_request_branch head-ref | Unattributed | Merged | [#37835](https://github.com/github/gh-aw/issues/37835) `bug` | 85 | **85** |
| [PR #37565](https://github.com/github/gh-aw/pull/37565) — Fix sample replay credential clobber | Unattributed | Merged | [#37545](https://github.com/github/gh-aw/issues/37545) `bug` | 85 | **85** |
| [PR #38185](https://github.com/github/gh-aw/pull/38185) — Add execcommandwithoutcontext linter | Unattributed | Merged | [#38143](https://github.com/github/gh-aw/issues/38143) `improvement` | 35 | **35** |
| [PR #36767](https://github.com/github/gh-aw/pull/36767) — [docs] Self-healing fixes 2026-06-04 | Unattributed | Closed | [#36275](https://github.com/github/gh-aw/issues/36275) `improvement` | 35 | 0 |
| [PR #36371](https://github.com/github/gh-aw/pull/36371) — [docs] Self-healing fixes 2026-06-02 | Unattributed | Closed | [#34686](https://github.com/github/gh-aw/issues/34686) `improvement` | 35 | 0 |
| [PR #35906](https://github.com/github/gh-aw/pull/35906) — [docs] Documentation update 2026-05-30 | Unattributed | Closed | [#35506](https://github.com/github/gh-aw/issues/35506) `improvement` | 35 | 0 |
| [PR #35330](https://github.com/github/gh-aw/pull/35330) — [docs] Self-healing fixes 2026-05-28 | Unattributed | Closed | [#34686](https://github.com/github/gh-aw/issues/34686) `improvement` | 35 | 0 |

---

## Unmapped Outcomes

_No unmapped in-scope outcomes._ All 7 analyzed PRs had at least one mapped label through their linked closing issues. (`cookie`, `deep-report` labels are unmapped but co-occur with mapped labels.)

---

## Interpretation

**Accepted outcome count (3)** appears low, but 993 PRs were excluded solely for lacking a `Closes #N` link — not for lack of agentic output.

**Impact Efficiency (0.00476)** is the better metric, but is artificially depressed: the denominator covers 80+ workflows over 180 days, most producing non-PR outcomes excluded from this analysis. Top AIC spenders (reporting/analysis workflows) collectively consumed ~20% of all AIC with zero attributed PR outcome value.

**Key findings:**
- **Bug fixes deliver the most value** (85/outcome): two compiler/workflow bug fixes account for 83% of total delivered value.
- **Four documentation PRs were rejected** — recurring pattern worth investigating in the documentation updater workflow.
- **Workflow attribution gap** is the single biggest blocker for useful per-workflow efficiency analysis.
- **Reporting/analysis workflows dominate AIC** with no PR-outcome return — this is expected, but cost visibility should drive prioritization.

---

## Data Quality

| Dimension | Status | Notes |
|---|---|---|
| PR linked-issue coverage | ⚠️ Weak | 99.3% of PRs excluded (993/1000). Only PRs with explicit `Closes #N` count. |
| Objective label mapping | ✅ Good | All 7 in-scope outcomes mapped. `cookie`, `deep-report` unmapped but coexist with mapped labels. |
| Workflow attribution | ❌ Missing | No `workflow_run_id` in PR dataset; `workflow-logs.json` covers only 10 runs from today. |
| Safe-output outcomes | ✅ Complete | 0 outcomes, confirmed deterministically. |
| AI Credits | ✅ Primary | 48 daily token-audit snapshots; total 43,055.27 AIC, 80 workflows. |
| Dataset completeness | ⚠️ Capped | Both PR datasets capped at 500 records; 180-day window may contain more. |

---

## Human Work

Because workflow attribution is absent from the PR dataset and `workflow-logs.json` covers only today's 10 runs, **all 500 merged PRs in the dataset appear human-authored** — likely an overcount (e.g., `[linter-miner]` and `[docs]` PRs are probably agentic).

| Metric | Value |
|---|---|
| Merged PRs in dataset | 500 (capped) |
| With linked closing issue | 3 |
| Without linked closing issue | 497 |
| Mapped to objective | 3 |
| Unmapped | 0 |

| Category | Map Value | PRs | Obj Value | Examples |
|---|---:|---:|---:|---|
| bug | 85 | 2 | 170 | [#37863](https://github.com/github/gh-aw/pull/37863), [#37565](https://github.com/github/gh-aw/pull/37565) |
| improvement | 35 | 1 | 35 | [#38185](https://github.com/github/gh-aw/pull/38185) |

> No AIC or Impact Efficiency computed for human work.

> Generated by [📊 Impact Efficiency Report](https://github.com/github/gh-aw/actions/runs/27701597488) · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Fobjective-impact-report%22&type=issues)