aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

feat(agent): tiered validation pipeline (Tiers 2–3)

Open
#490 0 comments 0 reactions 0 assignees View on GitHub
agent-runtime enhancement orchestration
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

**Context:** ROADMAP.md → Agent quality → Tiered validation pipeline
**Related:** #149 (ADR-013 closed), in-pipeline fix-up draft, evaluation pipeline draft

---

## Component

Agent (Python runtime)

## Describe the feature

Three post-agent tiers per roadmap:

1. **Tier 1 (already shipped):** tool validation—build/test/lint via agent post-hooks and PR body baselines. *This issue implements Tiers 2–3 only.*
2. **Tier 2:** code quality—DRY/SOLID/complexity analysis.
3. **Tier 3:** risk and blast radius analysis—feeds PR risk classifier and review routing (#425).

See ADR-013 tiered validation pyramid.

## Use case

Build/lint passing does not imply maintainable or low-risk changes. Operators want graduated validation before PR creation and to feed evaluation metrics.

## Proposed solution

1. Define tier interfaces in `agent/` with Blueprint opt-in per repo.
2. Tier 2: integrate existing linters/analyzers (eslint complexity, ruff, optional semgrep rules) with structured JSON output.
3. Tier 3: rule-based blast radius (files touched, public API surface, migrations)—output consumed by PR risk classifier draft.
4. Non-blocking advisory mode first; optional Cedar/policy gates later.
5. Emit validation results to `TaskEventsTable` and task record for dashboards.

## Other information

- ADR-013 documents the validation pyramid; this issue tracks **implementation**.
- Design context: `docs/decisions/ADR-013-tiered-validation-pyramid.md`, `docs/design/EVALUATION.md`.

- [ ] This might be a breaking change

Contributor guide

Open the contributing guide

Research direction

Start by reading docs/decisions/ADR-013-tiered-validation-pyramid.md and docs/design/EVALUATION.md, then inspect the agent/ interfaces and the TaskEventsTable references. Define how Tiers 2–3 integrate with Blueprint opt-in, structured advisory results, blast-radius output, and task records; done means these results can feed dashboards and the PR risk classifier.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.