[aw-failures] [aw] Failure Investigator Report — 2026-08-21 07:06 UTC (6h)

Fix the Claude Code CLI firewall egress path now — it crashed 2 workflows before a single token was logged, and every other Claude-engine workflow shares the same exposure until the allowlist is restored. File the Copilot CLI silent-exit gap next — it is losing root-cause data on every failure, silently. Leave the container vulnerability gate alone — it is working as designed, not a bug.

### Failure cluster table

| Cluster | Severity | Workflows | Runs | Signature | Status |
|---|---|---|---|---|---|
| Claude CLI egress blackout | **P0** | Daily VulnHunter Scan, Daily AgentRx Trace Optimizer | 2 | `api.anthropic.com` egress zeroed; `ERR_CONFIG` startup crash before structured logging | **New — sub-issue filed** |
| Copilot CLI silent exit | P1 | Daily Sub-Agent Model Resolution Audit, GitHub Remote MCP Authentication Test | 2 | Real tool activity (10+ tool calls, clean firewall), then bare `exit 1` with zero error text captured | New — documented here, not yet filed (issue-creation budget exhausted this cycle) |
| Container vuln gate | P2 | Daily Container Image Security Scan | 1 | `##[error]Critical vulnerabilities detected in container images.` | Working as intended — security gate correctly failing the build |
| Uninvestigated failures | Unknown | 15 other workflows (see below) | 15 | No log detail in this cycle's pre-fetch payload | Needs follow-up next cycle |

### Evidence

**P0 — Claude CLI egress blackout.** `audit-diff` of [§32451994270](https://github.com/github/gh-aw/actions/runs/32451994270) (failed) vs the last successful "Daily VulnHunter Scan" run, §32220853910 (2026-08-19), shows:
- `api.anthropic.com:443`: **54 allowed → 0 attempted/blocked.** Claude never reached Anthropic's API.
- Claude's own telemetry hosts vanished too: `o205451.ingest.us.sentry.io` (17→0), `otlp-gateway-prod-eu-west-2.grafana.net` (17→0).
- A new `(unknown)` domain was blocked once in the failed run — likely a new required host that isn't allowlisted.
- `token_usage` 33,824 → 0; run duration 10m44s → 5m39s (consistent with an immediate crash, not a hang).
- Same signature independently confirmed on [§32451257784](https://github.com/github/gh-aw/actions/runs/32451257784) ("Daily AgentRx Trace Optimizer") — same engine, same `ERR_CONFIG` text, same zero-token-usage pattern. This is cross-workflow, not workflow-specific.

P1 — Copilot CLI silent exit detail

`agenticworkflows audit` on [§32451181394](https://github.com/github/gh-aw/actions/runs/32451181394) ("Daily Sub-Agent Model Resolution Audit") shows a normal session — 32,216 tokens, 1 turn, 10 `task` calls, 6 `bash` calls, 0% firewall block rate — that only fails at the very end with a bare `##[error]Agent execution exited with code 1`, no richer diagnostic. Same shape on [§32451067175](https://github.com/github/gh-aw/actions/runs/32451067175) ("GitHub Remote MCP Authentication Test"). Unlike the Claude CLI cluster, this is a **post-execution/finalization failure with an observability gap** — the underlying error is never surfaced in logs even though the run clearly did real work.

15 uninvestigated failed runs (workflow / created_at)

- CLI Version Checker — §32450955372 — 2026-08-21T05:32:59Z
- Sighthound Security Scan — §32449943243 — 2026-08-21T05:16:28Z
- Issue Arborist — §32449932503 — 2026-08-21T05:16:18Z
- Daily GitHub Docs SEO Optimizer — §32448695306 — 2026-08-21T04:56:05Z
- Daily Go Test Parallelizer — §32447678169 — 2026-08-21T04:38:55Z
- Safe Output Health Monitor — §32446623477 — 2026-08-21T04:20:56Z
- Documentation Noob Tester — §32445151885 — 2026-08-21T03:56:34Z
- Designer Drift Audit — §32445120288 — 2026-08-21T03:56:00Z
- Daily AstroStyleLite Markdown Spellcheck — §32443500200 — 2026-08-21T03:28:21Z
- GPL Dependency Cleaner (gpclean) — §32443492271 — 2026-08-21T03:28:13Z
- DataFlow PR & Discussion Dataset Builder — §32441888201 — 2026-08-21T03:01:30Z
- Daily Go Test Parallelizer — §32441121542 — 2026-08-21T02:48:19Z
- Daily Regression Audit — §32440483938 — 2026-08-21T02:37:12Z
- AI Moderator — §32439801954 — 2026-08-21T02:25:34Z — **already tracked by #54242**
- Code Scanning Fixer — §32435866669 — 2026-08-21T01:19:04Z

Pre-fetch payload did not include `truncated_error_logs` for these; none re-fetched this cycle to stay within audit-call budget (capped at 2, both spent on the P0 cluster).

### Existing issue correlation

No open `agentic-workflows` issue matches either new cluster. Closest-looking candidates were checked and ruled out:
- #54242 ("Codex CLI exits code 1 twice... no error signature captured") — different engine (Codex vs Claude/Copilot), different workflow (AI Moderator), different mechanism (firewall-flavored exit-1, not a startup crash or finalization gap). Does cover run §32439801954 above.
- #54186 (cross-engine segfault, exit 139) — different exit code/crash mechanism entirely.

No issues were closed this cycle — no fresh evidence surfaced that any of the 18 open `agentic-workflows` issues are fixed or stale.

### Fix roadmap

**P0 — do this first:** Restore `api.anthropic.com` (and its Sentry/Grafana telemetry hosts) to the firewall allowlist; identify and explicitly allow the new `(unknown)` domain. See sub-issue below.

**P1 — do this next:** Capture the real error text for Copilot CLI finalization failures instead of swallowing it behind a bare `exit 1` — likely a log-capture/step-summary parsing gap, not a Copilot bug itself. File as a standalone sub-issue in the next cycle if it recurs.

**P2 — no action needed:** Container Image Security Scan gate failure is the gate working correctly (real critical vulnerabilities found). Monitor for repeated daily failures on the same CVE, which would indicate a stalled remediation rather than a workflow bug.

### Sub-issues created

- #54462 — [P0] Claude Code CLI startup crashes — Anthropic API egress blackout

**References:**
- [§32451994270](https://github.com/github/gh-aw/actions/runs/32451994270) — Daily VulnHunter Scan (P0)
- [§32451257784](https://github.com/github/gh-aw/actions/runs/32451257784) — Daily AgentRx Trace Optimizer (P0)
- [§32451181394](https://github.com/github/gh-aw/actions/runs/32451181394) — Daily Sub-Agent Model Resolution Audit (P1)

No source files or entry points are named. Start with sub-issue #54462 and compare the referenced failed and successful workflow runs, especially the firewall audit for the Claude CLI and the finalization logs for Copilot CLI; done means the P0 startup failure and P1 silent-exit diagnostic gap are resolved and verified in workflow runs.

Written by the indexing model from the issue text.