aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

feat(observability): Claude Code native OTel events + operator runbook

Open
#556 0 comments 0 reactions 0 assignees View on GitHub
documentation enhancement observability
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

Child of #245 (suggested sub-deliverable from the issue thread). This is the **per-model-call granularity** layer of the same attribution story: #245's custom harness spans + TaskEvents give task-level correlation; Claude Code's native OTel events add prompt/tool-level cost and token forensics without parsing unstructured logs.

## Background

The harness already wires Claude Code CLI telemetry behind `ENABLE_CLI_TELEMETRY=1` in `agent/src/runner.py` (`CLAUDE_CODE_ENABLE_TELEMETRY`, OTLP logs export, `OTEL_RESOURCE_ATTRIBUTES` with `task.id` / `repo.url` / `agent.model`). CDK leaves it **disabled** today (`ENABLE_CLI_TELEMETRY` commented in `cdk/src/stacks/agent.ts`) pending verification that the ADOT sidecar forwards OTLP **logs**, not just traces.

## Deliverables

1. **Enable + verify** — Turn on `ENABLE_CLI_TELEMETRY` in the AgentCore runtime env after confirming `claude_code.api_request` / `claude_code.tool_result` / `claude_code.tool_decision` records land in CloudWatch (application log group or AgentCore runtime log stream).
2. **Session correlation** — Ensure `task.id` on `OTEL_RESOURCE_ATTRIBUTES` joins to the #245 correlation envelope (`{task_id, user_id, repo, trace_id}`).
3. **Operator runbook** — Add `docs/guides/CLAUDE_CODE_TELEMETRY.md` (sync via Starlight) with:
- Event type reference (`claude_code.api_request` carries `cost_usd`, `input_tokens`, `output_tokens`, `duration_ms`)
- Copy-paste CloudWatch Logs Insights queries: per-task cost rollup, tool histogram, PreToolUse / hook blocks via `claude_code.tool_decision`
- Known limitations: Claude Code emits OTLP **events** (LogRecords), not spans — the Traces tab may be empty while Events/logs are populated; subagent attribution gaps
4. **Docs cross-links** — From `docs/design/OBSERVABILITY.md` and `docs/guides/COST_ATTRIBUTION.md` (complements #215 Bedrock billing tags).
5. **Optional** — Dashboard widgets for `cost_usd` by `task.id` when attributes are present.

## Acceptance criteria
- [ ] One dry-run task produces queryable `claude_code.api_request` records tagged with `task.id`
- [ ] Runbook queries documented and validated against a real log group
- [ ] `ENABLE_CLI_TELEMETRY` default documented (on vs opt-in) with sidecar forwarding notes
- [ ] `task.id` join to the #245 envelope verified

## Related
- Parent: #245 (correlation envelope — provides the `task.id` join key)
- #215 (Bedrock cost attribution), #237 (compliance export)
- Requires: ADOT sidecar OTLP **logs** forwarding (verification gate)

Contributor guide

Open the contributing guide

Research direction

Read agent/src/runner.py and cdk/src/stacks/agent.ts first, then verify whether the ADOT sidecar forwards OTLP logs to the relevant CloudWatch log group. Add docs/guides/CLAUDE_CODE_TELEMETRY.md and cross-links from docs/design/OBSERVABILITY.md and docs/guides/COST_ATTRIBUTION.md; done means a dry run shows queryable events tagged with task.id and the documented queries work against a real log group.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, typescript
Domain
cloud, documentation, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.