aws-samples / aws-samples/sample-autonomous-cloud-coding-agents
feat(observability): LLM-assisted trace analysis for failed tasks
- Dominant language
- TypeScript
- Stars
- 143
- Forks
- 46
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 20
Description
**Context:** ROADMAP.md → Agent quality → LLM-assisted trace analysis
**Related:** `--trace` (shipped), automated alert triage draft
---
## Component
API or orchestration
## Describe the feature
Automated **post-mortem** deep dive on failed task trajectories (S3 trace NDJSON + OpenTelemetry spans + `TaskEvents`) using an LLM to surface recurring reasoning and tool-use failure modes. Distinct from mid-run `bgagent ask` and optional status summarization.
## Use case
Operators debugging failure clusters spend hours reading traces. Platform teams need pattern detection across failures (e.g. repeated `git push` auth errors, hallucinated file paths).
## Proposed solution
1. Trigger on `task_failed` (optional Blueprint flag or operator-initiated `bgagent trace analyze`).
2. Lambda loads trace + events; calls Bedrock with structured output schema.
3. Persist analysis artifact (S3 + `TaskEvents` `trace_analysis` event).
4. Rate limits and cost caps per analysis.
5. Optional aggregation job for cross-task themes (weekly operator report).
## Other information
- Distinct from **automated alert triage** (security/ops alarms, not task post-mortems).
- Design context: `docs/design/OBSERVABILITY.md`, `docs/design/EVALUATION.md`.
- [ ] This might be a breaking change
Contributor guide
Research direction
Start with ROADMAP.md and docs/design/OBSERVABILITY.md and docs/design/EVALUATION.md, then trace the existing --trace flow and task_failed event path. Review the proposed bgagent trace analyze entry point and define how trace data, Bedrock analysis, persisted artifacts, rate limits, and optional aggregation fit together. Done means the feature scope and design are resolved for implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- backend-api-design, cloud, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100