aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

feat(ci): structured CI output — pytest JUnit XML + SARIF from eslint/ruff/semgrep (CA-06)

Open
#256 0 comments 0 reactions 0 assignees View on GitHub
ci-cd validation-loop
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

> **This is a finding from https://github.com/krokoko/cairn** (action item **CA-06**).

### Component

Tooling / CI

### Describe the feature

Emit **structured CI output** and upload it as artifacts + to GitHub code-scanning:

- **pytest** → JUnit XML (`--junitxml=test-reports/agent.xml`) — currently the agent suite is console-only with no structured artifact.
- **ESLint / ruff / semgrep** → **SARIF**, uploaded to GitHub code-scanning (`github/codeql-action/upload-sarif`).

This makes findings **agent-routable** — every failure carries file / line / rule / suggested-fix, the same property that makes the existing fail-on-mutation `repo.patch` artifact an exemplary agent-consumable signal.

### Use case

The feedback loop is at Level 2 (routable) but several lanes are stuck at Level 1: pytest has no JUnit reporter, and eslint/ruff/semgrep emit console text only. Without structured output, an agent (or the fix-and-repush loop) can't reliably parse *what* to fix and *where*. Structured, machine-routable findings are the prerequisite for closing the loop toward Level 3 (auto re-trigger on failure).

### Proposed solution

1. Add `--junitxml=test-reports/agent.xml` to the pytest invocation; upload via `actions/upload-artifact`.
2. Emit SARIF: `eslint -f @microsoft/sarif`, `ruff --output-format sarif`, semgrep `--sarif`.
3. Upload SARIF to GitHub code-scanning with `github/codeql-action/upload-sarif`.
4. Keep the existing JUnit (`jest-junit`) and lcov/cobertura coverage outputs.

### Acceptance criteria

- [ ] pytest emits JUnit XML uploaded as a CI artifact.
- [ ] ESLint, ruff, and semgrep emit SARIF.
- [ ] SARIF is uploaded to GitHub code-scanning and findings appear in the Security tab.
- [ ] No regression to the existing `jest-junit` / coverage artifacts.

### Other information

Source reports: `verification-report.md` (Feedback Loop Completeness), `verification-strategy.md` (Phase 1 #3; Feedback Loop Improvements), `ai-smells-gates-report.md` (Strengthen existing gates #3). Effort: **S**. Per ADR-003 this issue needs the `approved` label before work begins.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.