aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

Add moto-backed integration test for --trace upload/download round-trip

Open
#61 0 comments 0 reactions 0 assignees View on GitHub
agent-runtime cli validation-loop
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

> **Follow-up from [PR #52](https://github.com/aws-samples/sample-autonomous-cloud-coding-agents/pull/52)** — developer productivity investment identified during PR #52's L4 spike. Not a runtime bug; a test-tooling gap.

## Functional description

PR #52 shipped the `--trace` upload / download round-trip:

- **Upload** — `agent/src/telemetry.py` + `agent/src/pipeline.py` (agent writes to S3).
- **Download** — `cli/src/commands/trace.ts` + `cdk/src/handlers/get-trace-url.ts` (backend signs a URL; CLI downloads and gunzips).
- **Contract pinning** — `agent/tests/test_trace_key_contract.py` is a cross-language contract test that pins the literal key shape `traces//.jsonl.gz` across orchestrator, agent, handler, and CDK construct.

**The gap:** that contract test is a strong drift detector for the *key shape* but does **NOT** exercise the actual round-trip (agent writes → handler presigns → CLI downloads + gunzips). If a developer accidentally breaks the interaction between these three layers, the only way to catch it is to deploy to AWS and try it by hand.

Full CDK-stack E2E isn't feasible locally — Bedrock AgentCore has no LocalStack / moto equivalent. A moto-backed data-plane harness is the pragmatic middle-ground: fake AWS running in-process, exercising real code paths in under 10 seconds.

**Who cares:** contributors and maintainers. No user-facing impact. This is a developer-velocity investment.

## Technical plan

Create a new integration test module:

**File:** `agent/tests/integration/test_trace_roundtrip.py`

**Fixtures:**
- moto-backed S3 bucket matching `TraceArtifactsBucket` properties (7-day lifecycle, SSE, blocked public access).
- moto-backed DynamoDB `TaskTable` with the real GSI schema.

**Test flow:**
1. Seed a fake `TaskRecord` with `trace=true`, `user_id`, `task_id`.
2. Invoke `upload_trace_to_s3` with a fabricated trajectory.
3. Directly invoke the `get-trace-url` handler logic — either port its core to Python, or drive it via a lightweight shim that reads `trace_s3_uri` from mocked `TaskTable` and signs a URL.
4. Use the CLI's download path (subprocess `bgagent trace download` against a mocked API Gateway, or direct-invoke the CLI command module) to verify gunzip round-trips and JSONL reconstitutes.

**Runtime:** `mise //agent:test:integration` or `pytest -m integration`.

**Estimated effort:** 1-2 days for a minimal version. This is greenfield test scaffolding — no existing LocalStack / moto infrastructure in the repo today.

## Acceptance criteria

- One end-to-end test exercising write → read → CLI decode.
- Runs in <10 seconds.
- CI wiring via explicit opt-in marker (e.g. `pytest -m integration`) so it doesn't slow the default test loop.
- `agent/tests/integration/README.md` explaining how to run locally and what the harness does / doesn't cover.

## Out of scope

- CDK synth / deploy in tests (too heavy; AgentCore not mockable).
- Cognito auth flow (already covered by unit tests in CDK + CLI).
- Cross-user authorization tests (covered by handler unit tests).

## References

- `agent/src/telemetry.py` — trace upload
- `cli/src/commands/trace.ts` — CLI download
- `cdk/src/handlers/get-trace-url.ts` — backend URL signing
- `agent/tests/test_trace_key_contract.py` — existing contract test (this issue *complements*, not replaces)
- PR #52 L4 spike evaluation (this was scoped out of PR #52 for delivery, not dropped)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.