[agent-deep-dive] Agent deep dive: gh-aw-docs-drift.lock.yml - 2026-09-10

## Agent Deep Dive: `gh-aw-docs-drift.lock.yml`
**Date:** 2026-09-10
**Runs analyzed:** 20 total (0 success, 20 failure, 0 other)

## Run Outcomes
| Run | Date | Conclusion |
|-----|------|------------|
| [29840735847](https://github.com/elastic/ai-github-actions/actions/runs/29840735847) | 2026-07-21 | failure |
| [29840045629](https://github.com/elastic/ai-github-actions/actions/runs/29840045629) | 2026-07-21 | failure |
| [29836279865](https://github.com/elastic/ai-github-actions/actions/runs/29836279865) | 2026-07-21 | failure |
| [29836122589](https://github.com/elastic/ai-github-actions/actions/runs/29836122589) | 2026-07-21 | failure |
| [23928111464](https://github.com/elastic/ai-github-actions/actions/runs/23928111464) | 2026-04-03 | failure |
| [23888691648](https://github.com/elastic/ai-github-actions/actions/runs/23888691648) | 2026-04-02 | failure |
| [23862505883](https://github.com/elastic/ai-github-actions/actions/runs/23862505883) | 2026-04-01 | failure |
| [23094261722](https://github.com/elastic/ai-github-actions/actions/runs/23094261722) | 2026-03-14 | failure |
| [23093951466](https://github.com/elastic/ai-github-actions/actions/runs/23093951466) | 2026-03-14 | failure |
| [22262096220](https://github.com/elastic/ai-github-actions/actions/runs/22262096220) | 2026-02-21 | failure |
| [22262092733](https://github.com/elastic/ai-github-actions/actions/runs/22262092733) | 2026-02-21 | failure |
| [22262042033](https://github.com/elastic/ai-github-actions/actions/runs/22262042033) | 2026-02-21 | failure |
| [22261977222](https://github.com/elastic/ai-github-actions/actions/runs/22261977222) | 2026-02-21 | failure |
| [22261894612](https://github.com/elastic/ai-github-actions/actions/runs/22261894612) | 2026-02-21 | failure |
| [22261801845](https://github.com/elastic/ai-github-actions/actions/runs/22261801845) | 2026-02-21 | failure |
| [22261700095](https://github.com/elastic/ai-github-actions/actions/runs/22261700095) | 2026-02-21 | failure |
| [22261697405](https://github.com/elastic/ai-github-actions/actions/runs/22261697405) | 2026-02-21 | failure |
| [22261373775](https://github.com/elastic/ai-github-actions/actions/runs/22261373775) | 2026-02-21 | failure |
| [22261294666](https://github.com/elastic/ai-github-actions/actions/runs/22261294666) | 2026-02-21 | failure |
| [22261290949](https://github.com/elastic/ai-github-actions/actions/runs/22261290949) | 2026-02-21 | failure |

## Failure Analysis
All analyzable failed-run artifacts in `/tmp/gh-aw/deep-dive/logs//logs.zip` are not ZIP archives; each contains the same GitHub API 404 payload instead:

`{"message":"Not Found","documentation_url":"https://docs.github.com/rest/actions/workflow-runs#download-workflow-run-logs","status":"404"}`

Observed on every downloaded failed run artifact (10/10): 22262096220, 23093951466, 23094261722, 23862505883, 23888691648, 23928111464, 29836122589, 29836279865, 29840045629, 29840735847.

Likely cause: the deep-dive pre-step pulls old failed runs whose log archives are no longer available under repository log retention, then still proceeds to analysis with empty/invalid data.

## Tool Call Patterns
- Average turns per run: **N/A (agent step logs unavailable due 404 log downloads)**
- Most-called tools: **N/A**
- Anomalies: **All sampled failed runs share the same pre-analysis failure mode (missing logs), so downstream tool-trace analysis cannot execute.**

## Behavioral Observations
The target workflow is intended to perform behavioral deep dives, but in this sample it cannot inspect any run internals because the preloaded artifacts are missing. The workflow still asks for detailed tool-call/turn analysis, creating an impossible task when `errors.json` reports `total_files_scanned: 0` and `total_matches: 0`.

## Recurring Patterns
1. 20/20 recent runs conclude as failure in `runs.json`.
2. 10/10 downloaded failed-run log artifacts resolve to identical GitHub 404 payloads instead of ZIP logs.
3. Error extraction has no input files (`errors.json`: `total_files_scanned=0`).

## Prompt Improvement Suggestions
1. **Gate on log availability before analysis prompt execution**
- Observed problem: analysis step runs even when every failed-run log fetch returns 404 and no text logs exist.
- Prompt/workflow change: after log download + extraction, add a hard check (e.g., if extracted txt file count == 0 then call `noop` with explicit reason and stop).
- Expected improvement: avoids impossible deep-dive prompts and eliminates noisy/non-actionable reports.

2. **Select runs with available logs instead of fixed recent-N**
- Observed problem: fixed recent run window included only runs with unavailable log archives.
- Prompt/workflow change: iterate over a larger recent set and keep only runs whose log download returns a real ZIP (or whose jobs/logs API calls succeed), then analyze up to N valid runs.
- Expected improvement: higher chance of obtaining analyzable traces; better failure/root-cause insights.

3. **Fallback to job-level logs when run archive endpoint returns 404**
- Observed problem: single-endpoint dependency on `actions/runs/{run_id}/logs` leaves no fallback path.
- Prompt/workflow change: on 404 for run logs, list jobs for the run and fetch per-job logs (where available), then feed those into `extract-log-errors.py`.
- Expected improvement: recovers partial observability and enables tool-call/failure tracing instead of N/A outputs.

> Generated by [Internal: Agent Deep Dive](https://github.com/elastic/ai-github-actions/actions/runs/34491890216) · gpt53codex · 31.2 AIC · ⌖ 3.89 AIC · ⊞ 23.1K · [◷](https://github.com/search?q=repo%3Aelastic%2Fai-github-actions+is%3Aissue+%22gh-aw-workflow-call-id%3A+elastic%2Fai-github-actions%2Fagent-deep-dive%22&type=issues)
> - [x] expires on Sep 24, 2026, 2:59 PM UTC

Start with gh-aw-docs-drift.lock.yml and the log-download and extraction steps described in the report, including extract-log-errors.py. Reproduce the 404 response for an unavailable run archive and inspect how empty extracted logs are handled. Done means the workflow avoids analysis when no logs are available and follows the selected fallback or run-selection behavior with actionable output.

Written by the indexing model from the issue text.