aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

feat(observability): LLM-assisted trace analysis for failed tasks

Open
#458 0 comments 0 reactions 0 assignees View on GitHub
agent-runtime enhancement observability
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

**Context:** ROADMAP.md → Agent quality → LLM-assisted trace analysis
**Related:** `--trace` (shipped), automated alert triage draft

---

## Component

API or orchestration

## Describe the feature

Automated **post-mortem** deep dive on failed task trajectories (S3 trace NDJSON + OpenTelemetry spans + `TaskEvents`) using an LLM to surface recurring reasoning and tool-use failure modes. Distinct from mid-run `bgagent ask` and optional status summarization.

## Use case

Operators debugging failure clusters spend hours reading traces. Platform teams need pattern detection across failures (e.g. repeated `git push` auth errors, hallucinated file paths).

## Proposed solution

1. Trigger on `task_failed` (optional Blueprint flag or operator-initiated `bgagent trace analyze`).
2. Lambda loads trace + events; calls Bedrock with structured output schema.
3. Persist analysis artifact (S3 + `TaskEvents` `trace_analysis` event).
4. Rate limits and cost caps per analysis.
5. Optional aggregation job for cross-task themes (weekly operator report).

## Other information

- Distinct from **automated alert triage** (security/ops alarms, not task post-mortems).
- Design context: `docs/design/OBSERVABILITY.md`, `docs/design/EVALUATION.md`.

- [ ] This might be a breaking change

Contributor guide

Open the contributing guide

Research direction

Start with ROADMAP.md and docs/design/OBSERVABILITY.md and docs/design/EVALUATION.md, then trace the existing --trace flow and task_failed event path. Review the proposed bgagent trace analyze entry point and define how trace data, Bedrock analysis, persisted artifacts, rate limits, and optional aggregation fit together. Done means the feature scope and design are resolved for implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
backend-api-design, cloud, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.