aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

agent: Fail-fast preflight verify step before run_agent

Offen
#563 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
agent-runtime enhancement validation-loop
Vorherrschende Sprache
TypeScript
Sterne
143
Forks
46
Ø Merge
3 T. 10 Std.
Gemergte PRs (30 T.)
24

Beschreibung

## Component

Agent (Python runtime)

## Describe the feature

Support an explicit **`verify_build` or `verify_lint` step before the first `run_agent`** that **fails the task without invoking the LLM** when the repo is already broken.

```yaml
steps:
- { kind: clone_repo, name: setup }
- { kind: verify_build, name: preflight, gate: strict, on_failure: fail }
- { kind: run_agent, name: implement }
- { kind: verify_build, name: build, gate: regression_only }
- { kind: ensure_pr, name: open_pr, strategy: create }
```

## Use case

**What exists today (not sufficient):**

- `clone_repo` / `setup_repo` already runs `mise run build` and `mise run lint` deterministically and records `build_before` / `lint_before` in `RepoSetup`.
- If the initial build fails, the task **still proceeds to `run_agent`** — failure is noted in setup notes and prompts tell the agent to run build, but there is no workflow-level fail-fast gate.
- Post-agent `verify_build` runs only after the agent consumes tokens.

**Gap:** Operators need a declarative, deterministic preflight gate — not prompt guidance — to skip LLM execution when the tree is already red.

Orchestrator pre-flight ([ARCHITECTURE.md](https://github.com/aws-samples/sample-autonomous-cloud-coding-agents/blob/main/docs/design/ARCHITECTURE.md)) covers GitHub reachability; this covers **repo health after clone**.

## Proposed solution

1. **Validator** — If a `verify_*` step appears before `run_agent`, `gate` must be `strict` or `regression_only` (not `informational`); record baseline in `workflow_state.json` for post-agent `regression_only` compare.
2. **Runner** — Step order already supported; add test proving preflight failure never invokes `run_agent`.
3. **Cost** — `clone_repo` already runs build; consider skipping duplicate full build in clone when a preflight `verify_build` immediately follows, or document opt-in on workflows only.
4. **Docs** — WORKFLOWS.md preflight pattern + cost trade-off.
5. **Opt-in** — Enable on `coding/new-task-v1` only after cold-start cost measurement.

## Other information

- Forbidden when `requires_repo: false` (existing validator rule).
- Related: #457 (post-agent fix loops — complementary).

## Acknowledgements

- [ ] I may be able to implement this feature
- [ ] This might be a breaking change

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Lies docs/design/ARCHITECTURE.md und WORKFLOWS.md und verfolge dann die Validator- und Runner-Einstiegspunkte verify_* rund um clone_repo und run_agent. Füge Abdeckung für einen fehlschlagenden Preflight hinzu, der run_agent nie aufruft, erfasse Baselines für workflow_state.json und dokumentiere das Preflight-Muster sowie den Zielkonflikt bei den Build-Kosten.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python, typescript
Bereich
backend, documentation, tooling
Issue-Typ
Feature
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
55/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.