elastic / elastic/ai-github-actions
Add retry configuration for safe-outputs threat-detection to avoid silent security-check failures
- Dominant language
- Python
- Stars
- 11
- Forks
- 16
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 31
Description
## Summary
Our `PR Review` agentic workflow (`.github/workflows/gh-aw-pr-review.md` → compiled `gh-aw-pr-review.lock.yml`) uses `safe-outputs` with the default `threat-detection` configuration. We hit a run where the detection step failed with `reason=invalid_report_exhausted exit=2`, even though the underlying Copilot engine had actually invoked the `threat_detection_result` tool with a valid verdict (see [github/gh-aw-threat-detection#1014](https://github.com/github/gh-aw-threat-detection/issues/1014) for the upstream root-cause analysis).
Reference run: https://github.com/elastic/docs-builder/actions/runs/33497222727 (job "run / detection")
## Problem
By design, `gh-aw`'s `threat-detection` defaults `max-retries` to **0** for the detection engine (unlike the main agent, which defaults to 3 retries), specifically because detection is meant to be a bounded scan. However, this means any transient hiccup in the detection engine (harness/tool-call race conditions, momentary API issues, etc.) causes the **entire security check to fail outright with zero retries**, and — because our workflow has `continue-on-error: true` on the detection step (required so a detection failure doesn't block the whole workflow) — the failure is silently swallowed. The overall workflow run reports "success" even though the threat-detection safety net never actually produced a verdict.
This means we currently have **no resilience** against a class of transient detection failures, and no visibility when they occur (the workflow just looks green).
## Request
Please add explicit retry configuration to our `safe-outputs.threat-detection` frontmatter across our `gh-aw`-based workflows (starting with `gh-aw-pr-review`), e.g.:
```yaml
safe-outputs:
threat-detection:
engine:
harness:
max-retries: 2
```
Additionally, please consider:
1. Auditing all reusable workflows in this repo that configure `threat-detection` and adding a sensible non-zero retry count to each, rather than relying on the `gh-aw` default of 0.
2. Adding alerting/visibility (e.g., a step summary annotation or a follow-up issue) when `continue-on-error: true` masks a detection failure, so we don't silently lose the security check without noticing.
3. Documenting this trade-off (continue-on-error + zero retries = silent security-check gaps) in our internal `gh-aw` workflow authoring guidelines.
## References
- gh-aw threat-detection config docs: https://github.github.com/gh-aw/reference/threat-detection/
- Upstream bug report (root cause of the specific failure): github/gh-aw-threat-detection#1014
Contributor guide
Assessment
This issue has not been assessed yet.