feat(traces): compare two AgentCore traces from the CLI
- Dominant language
- TypeScript
- Stars
- 283
- Forks
- 95
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 183
Description
## Problem
`agentcore traces list` finds traces and `agentcore traces get` exports raw
CloudWatch records, but there is no human-friendly way to compare two agent
invocations.
Users testing runtime versions or named endpoints must manually download JSON,
inspect CloudWatch’s incomplete single-span view, and calculate latency/token
differences themselves. This makes performance regression testing impractical.
## Proposed command
```bash
agentcore traces compare \
--runtime \
[--since
Example output:
```
Trace comparison: baseline abc123 → candidate def456
Metric Baseline Candidate Delta
End-to-end latency 10.00s 6.33s -3.66s (-36.6%)
LLM latency 7.36s 3.67s -3.69s (-50.1%)
Tool latency 2.26s 2.58s +0.32s (+14.3%)
LLM calls 2 2 0 (0.0%)
Tool calls 1 1 0 (0.0%)
Input tokens 151,266 2,848 -148,418 (-98.1%)
Output tokens 595 300 -295 (-49.6%)
Total tokens 151,861 3,148 -148,713 (-97.9%)
Baseline model(s): us.anthropic.claude-haiku-4-5-20251001-v1:0
Candidate model(s): us.anthropic.claude-haiku-4-5-20251001-v1:0
```
## Required behavior
- Resolve the runtime/project target using the existing traces command
conventions.
- Fetch structured span records from CloudWatch directly.
- Use the POST /invocations server span for end-to-end latency when present.
If absent, fall back to earliest span start through latest span end and label
that fallback clearly.
- Report LLM and tool time separately using existing GenAI span attributes.
- Avoid double-counting nested provider spans. For example, a Strands internal
LLM span and its nested Bedrock client span represent the same model call.
- Include LLM/tool call counts and input/output/total token counts when
available.
- Show absolute and percentage deltas; handle a zero baseline safely.
- Provide stable machine-readable output with --json.
- Fail clearly if either trace cannot be found or has no usable timed spans.
- Do not claim a “critical path” calculation: CloudWatch trace parent/child
relationships may be incomplete.
- Show comparability warnings when observable characteristics differ, such as
LLM-call count, tool-call count, or token usage. The CLI cannot prove that
## Scope
This is a trace-analysis and latency-comparison feature. It is independent of:
- runtime endpoint selection for agentcore invoke
- AgentCore Gateway A/B-test infrastructure
- SigNoz or other third-party OTEL observability backends
### Acceptance Criteria
- agentcore traces compare works for two trace IDs from the same runtime.
- It produces the table above or equivalent concise terminal output.
- --json returns documented structured data suitable for CI benchmarking.
- Unit tests cover root-span selection, fallback timing, nested-span
de-duplication, token aggregation, deltas, warnings, and missing-trace
errors.
- Existing traces list and traces get behavior remains unchanged.
### Additional Context
_No response_
Contributor guide
Research direction
Start with the existing `agentcore traces list` and `agentcore traces get` commands, their CloudWatch trace/span sources, target-resolution conventions, and GenAI span attributes. Then use the requested unit-test areas—root-span selection, fallback timing, nested-span de-duplication, token aggregation, deltas, warnings, and missing-trace errors—as the definition of done, while preserving existing list/get behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cli, cloud, observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100