aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

(cli): `bgagent stats` for agent contribution and quality metrics

Open
#412 0 comments 0 reactions 0 assignees View on GitHub
cli cost-finops enhancement observability
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

## Component

Scripts / CLI

## Describe the feature

Add `bgagent stats` to report aggregate outcomes over a time window: tasks submitted, PRs opened, merge rate, median revision cycles, CI pass rate, cost totals—filterable by repo and user.

## Use case

Operators adopting background agents need **outcome metrics**, not task counts alone. Roadmap **Evaluation pipeline** and operator dashboard cover backend aggregation; CLI users and platform teams want a quick, scriptable view for adoption reviews and executive summaries.

## Proposed solution

1. `GET /stats` (or query Tasks table + webhook-derived merge state) returning JSON aggregates.
2. CLI: `bgagent stats [--repo] [--since 30d] [--format table|json]`.
3. Reuse PR outcome signals from roadmap **PR outcome tracking** when available; v1 can use task terminal states + GitHub API sampling with documented limitations.
4. Document proxy vs. customer-impact metrics (merge rate is proxy; optional manual tags later).

## Other information

- Complements roadmap **Validation and risk analytics** and control panel—not a duplicate if scoped to CLI/API summary.
- Auth: same Cognito constraints as `list`/`status`.

## Acknowledgements

- [ ] I may be able to implement this feature
- [ ] This might be a breaking change

## Acceptance criteria

- [ ] API endpoint documented in `API_CONTRACT.md` with stable response shape mirrored in `cli/src/types.ts`.
- [ ] CLI prints table and JSON; respects `--verbose` for query details.
- [ ] Tests in `cli/test/` with mocked API client.
- [ ] User guide explains metric definitions and limitations.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the acceptance criteria in API_CONTRACT.md and the response types in cli/src/types.ts, then inspect the existing CLI API-client tests under cli/test/. Done means a documented stable stats endpoint, table and JSON output with the requested filters and verbose details, mocked tests, and a user guide covering metric definitions and limitations.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, github, typescript
Domain
analytics, api, cli, documentation, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.