[agent-deep-dive] Agent deep dive: gh-aw-agent-suggestions.lock.yml - 2026-09-14

## Agent Deep Dive: `gh-aw-agent-suggestions.lock.yml`
**Date:** 2026-09-14
**Runs analyzed:** 6 total (0 success, 6 failure, 0 other)

## Run Outcomes
| Run | Date | Conclusion |
|-----|------|------------|
| [29840746803](https://github.com/elastic/ai-github-actions/actions/runs/29840746803) | 2026-07-21 | failure |
| [29836296207](https://github.com/elastic/ai-github-actions/actions/runs/29836296207) | 2026-07-21 | failure |
| [29836106127](https://github.com/elastic/ai-github-actions/actions/runs/29836106127) | 2026-07-21 | failure |
| [23928107230](https://github.com/elastic/ai-github-actions/actions/runs/23928107230) | 2026-04-03 | failure |
| [23888692954](https://github.com/elastic/ai-github-actions/actions/runs/23888692954) | 2026-04-02 | failure |
| [23862505560](https://github.com/elastic/ai-github-actions/actions/runs/23862505560) | 2026-04-01 | failure |

## Failure Analysis
For every failed run, the preloaded artifact at `/tmp/gh-aw/deep-dive/logs//logs.zip` is not a ZIP archive; it contains:

```json
{"message":"Not Found","documentation_url":"https://docs.github.com/rest/actions/workflow-runs#download-workflow-run-logs","status":"404"}
```

Because all run logs are unavailable, the agent-step failure point, exact in-run error text, and tool-level sequence cannot be reconstructed from the provided data.

Likely cause: run logs were unavailable at collection time (retention expiry, deleted run artifacts, or log-download access mismatch), causing the deep-dive collector to persist 404 payloads instead of agent logs.

## Tool Call Patterns
- Average turns per run: **Not observable** (no agent step logs available)
- Most-called tools: **Not observable**
- Anomalies: The same `404 Not Found` log-download result occurred for **all 6/6 runs**, indicating a systemic data-collection blind spot.

## Behavioral Observations
- The workflow has a severe reliability signal at the run level (0% success in the analyzed set).
- Observability is currently insufficient for root-cause analysis because no run includes recoverable agent step logs.
- The deep-dive package (`runs.json`, `errors.json`, `logs/`) contains run metadata but no actionable execution trace for any failed run.

## Recurring Patterns
1. **Run-level recurrence:** all six recent sampled runs concluded `failure`.
2. **Telemetry recurrence:** all six corresponding log downloads resolved to the same `404 Not Found` payload.

## Prompt Improvement Suggestions
1. **Add an explicit "data sufficiency gate" before behavioral analysis.**
- Observed problem: deep-dive artifacts lacked agent logs for all runs, preventing turn/tool-call analysis.
- Prompt change: require the agent to verify that each `logs.zip` is a valid archive and that at least one agent-step log is present before proceeding.
- Expected improvement: avoids producing speculative deep-dives and surfaces observability failures immediately.

2. **Require fallback evidence collection when logs are missing.**
- Observed problem: with only 404 payloads, failure-step details were unavailable.
- Prompt change: instruct collection of workflow job/step conclusions and annotations (from run metadata APIs) when logs are inaccessible.
- Expected improvement: still yields concrete failure-mode diagnostics without raw logs.

3. **Bias sampling toward newest runs within confirmed log-retention window.**
- Observed problem: older runs are more likely to have inaccessible logs; this can collapse the entire analysis.
- Prompt change: fetch and analyze only runs whose age is within repository log-retention policy, and stop with an explicit `noop` if that constraint cannot be met.
- Expected improvement: higher chance of complete traces, fewer wasted deep-dive cycles.

4. **Fail fast on all-404 deep-dive datasets.**
- Observed problem: dataset appeared complete structurally but was unusable semantically.
- Prompt change: if all failed-run log downloads return non-archive payloads (e.g., 404 JSON), report a dedicated observability issue and skip workflow-behavior conclusions.
- Expected improvement: cleaner separation between workflow reliability findings and telemetry-pipeline failures.

> Generated by [Internal: Agent Deep Dive](https://github.com/elastic/ai-github-actions/actions/runs/34858483535) · gpt53codex · 15.9 AIC · ⌖ 3.72 AIC · ⊞ 23.1K · [◷](https://github.com/search?q=repo%3Aelastic%2Fai-github-actions+is%3Aissue+%22gh-aw-workflow-call-id%3A+elastic%2Fai-github-actions%2Fagent-deep-dive%22&type=issues)
> - [x] expires on Sep 28, 2026, 3:05 PM UTC