aws-samples / aws-samples/sample-autonomous-cloud-coding-agents
Tooling: Harness mutation regression suite for workflow and policy changes
- 主要言語
- TypeScript
- スター
- 146
- フォーク
- 46
- 平均マージ
- 3日 10時間
- マージ済み PR(30日)
- 24
説明
> **Roadmap:** Exposed project specifications; workflow promotion gate
> **Priority:** P2
## Component
Other
## Describe the feature
CI regression suite that treats **harness mutations** (workflow YAML, Cedar policies, contract fixtures) like safety-critical code changes. On changes to `agent/workflows/**`, `agent/policies/**`, or `contracts/**`, run golden trajectory fixtures and cedar-parity tests; fail on event-order, policy decision, or terminal-state regressions.
Implements arXiv:2605.18747 §5.2.3: harness edits require held-out regression suites and rollback discipline.
## Use case
- **Contributors** changing workflows or Cedar rules get immediate feedback before deploy.
- **Workflow promotion** (`draft → production`) gains an automated gate beyond unit tests.
- **Operators** forking workflows have a template for their own regression fixtures.
## Proposed solution
1. Depends on golden PEV trajectory fixtures (`contracts/harness-trajectories/` — separate issue).
2. GitHub Actions job (or extend existing build workflow):
- Trigger paths: `agent/workflows/**`, `agent/policies/**`, `contracts/**`
- Run: workflow-validation, cedar-parity, golden trajectory diff
- Output: fixture diff on failure
3. Document operator expectation: production workflow changes require green regression suite.
### Acceptance criteria
- [ ] CI job triggers on harness-relevant path changes
- [ ] Fails with actionable diff when golden trajectory or cedar-parity regresses
- [ ] Documented in `CONTRIBUTING.md` or developer guide
- [ ] Works without live Bedrock (mocked agent/orchestrator responses)
## Other information
- **Blocked by (soft):** Conformance golden PEV trajectories issue
- **Related:** Workflow promotion gate in `WORKFLOWS.md`
- **Paper:** arXiv:2605.18747 §3.5, §5.2.3 governed harness mutation
## Acknowledgements
- [ ] I may be able to implement this feature
- [ ] This might be a breaking change
コントリビューションガイド
調査の方向性
agent/workflows/**、agent/policies/**、contracts/** 配下の変更を調査し、その後、既存の build workflow と、WORKFLOWS.md にある関連するプロモーションガイダンスを確認してください。別の contracts/harness-trajectories issue は依存関係です。完了には、パスによってトリガーされる CI、workflow-validation、cedar-parity、golden trajectory のチェックを、実用的な diff、モックされたレスポンス、および CONTRIBUTING.md または開発者ガイドのドキュメントとともに実施することが必要です。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- github-actions, yaml
- 領域
- ci-cd, devops, testing
- issue の種類
- 機能追加
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 静か
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 48/100