RFC: Governance planes — analytics and compliance export

### Primary area

Cross-cutting / multiple

### Related issue or feature request

Roadmap: [Cost dashboard and export API](https://github.com/aws-samples/sample-autonomous-cloud-coding-agents/blob/main/docs/guides/ROADMAP.md), [Immutable audit export](https://github.com/aws-samples/sample-autonomous-cloud-coding-agents/blob/main/docs/guides/ROADMAP.md).

### Summary

Governance-relevant data already exists in ABCA but is **fragmented** across `TaskEvents`, the Tasks table, CloudWatch, X-Ray, Bedrock invocation logs, and Cedar HITL metadata. Operators stitch these together manually for leadership reporting, chargeback, and security investigations.

This RFC adds a **governance layer** complementary to operational observability (see [OBSERVABILITY.md](https://github.com/aws-samples/sample-autonomous-cloud-coding-agents/blob/main/docs/design/OBSERVABILITY.md)):

1. **Analytics plane** — Time-bucketed usage and outcome metrics (`GET /v1/analytics/*`, admin CSV), with `cost_provenance` until Bedrock tag attribution ships.
2. **Compliance plane** — Versioned `abca.audit.v1` export via dedicated `compliance-export` stream consumer → Firehose → customer S3 (optional pull API).

Operational telemetry (dashboards, alarms, `bgagent watch`) stays unchanged. Analytics and compliance are **downstream consumers** of `TaskEvents`, not a third write path.

**Value proposition:** Teams need to **prove** who ran what, **measure** cost and outcomes at scale, and **improve** prompts/policy from aggregated evidence—not only inspect one failed trace. See the relationship-to-observability section in the full RFC.

### Use case and motivation

| Persona | Gap today |
|---------|-----------|
| Platform owner / EM | No stable leadership API for tasks/cost per repo |
| FinOps | Weak Bedrock roll-ups; task-record cost only |
| Security / GRC | No SIEM export contract for policy denies |
| Prompt / policy owner | `prompt_version` exists; evaluation mostly manual |
| Auditor | DDB TTL; no WORM / long-term export |

**Problems:** no programmatic reporting contract; scattered compliance data; over-broad CloudWatch access for simple governance questions; retention mismatch (90-day hot vs. longer aggregates/archives).

### Proposal

**Three planes:** Operational (existing) | Analytics (hours, roll-ups) | Compliance (event grain, export).

**v1 decisions:**

| Topic | Decision |
|-------|----------|
| Workspace | One CDK stack = one workspace; no `org_id` in v1 |
| Compliance consumer | **Separate** Lambda + DLQ on `TaskEvents` stream—not `fanout-task-events` |
| Cost | `cost_provenance`: `task_record` (default) or `bedrock_cur` when available |
| Roll-ups | 13-month TTL; counts/sums only; admin purge API per `user_id` |
| ADR | **ADR-014** when this issue is `approved` |

**Analytics (high level):**

- `GET /v1/analytics/usage` with `group_by=day|week`, dimensions: `user_id`, `repo`, `task_type`, `prompt_version`.
- **Beta** until phase 1b roll-up table; phase 1a on-read spike: 30-day max, admin scope only.
- Non-goals: LOC generated, tool acceptance rate as KPIs.

**Compliance (high level):**

- Schema `abca.audit.v1` with required **`correlation`** (`session_id`, `trace_id`, `trace_s3_uri`, `bedrock_log_ref`).
- Default tier `metadata_only`; `full_content` = pointers only (Bedrock logs / S3 trace).
- Firehose → customer S3; hourly SHA-256 manifests; Object Lock optional.
- **Prerequisite:** export enabled in prod for audits beyond `taskRetentionDays`.

**UX before → after:**

- Manager: Log Insights scripts → `GET /v1/analytics/usage` or CSV.
- Security: per-task DDB export → SIEM via Firehose + `correlation` fields.
- FinOps: provisional task cost → CUR join when `cost_provenance=bedrock_cur`.
- Task submitters: unchanged unless granted analytics roles.

**Phased delivery:**

| Phase | Deliverable |
|-------|-------------|
| 0 | RFC approved; ADR-014 |
| 1a | Analytics API beta (on-read, 30d) |
| 1b | Roll-up table; beta lifted |
| 1c | User erasure + 13mo TTL |
| 2a | `compliance-export` + Firehose + manifests |
| 2b | Optional pull API |
| 3 | Control panel widgets |
| 4 | `redacted_content` / `full_content` tiers |

**Full design** (diagrams, API shapes, coverage matrix, acceptance criteria): `docs/design/RFC_GOVERNANCE_PLANES.md`.

### Out of scope

- Replacing CloudWatch / X-Ray / GenAI Observability consoles
- Sub-second real-time analytics
- Built-in SIEM product (export contract + delivery only)
- Customer source code in exports (`pr_url` only)
- Multi-account SaaS; per-user `org_id` in v1
- Prompt A/B assignment (feeds future work only)

### Potential challenges

| Risk | Mitigation |
|------|------------|
| PII/secrets in metadata | Redaction transformer; CI scans |
| DDB scan cost (1a) | 30-day cap; GA blocked on 1b roll-ups |
| FinOps misreads cost | `cost_provenance` on every bucket |
| Fan-out coupling | Separate compliance Lambda + DLQ |
| Schema drift | `abca.audit.v1` + contract tests |
| Stream retries | Idempotent `record_id` |
| `full_content` too open | Separate scope + break-glass logging |

**Open questions:** roll-up storage DDB vs Parquet; GitHub outcome webhook vs poll; KMS-signed manifests; Tasks stream vs orchestrator-only `task_summary`.

### Dependencies and integrations

- **AWS:** DynamoDB Streams, Lambda, Firehose, S3, KMS, CloudTrail
- **Auth:** Cognito scopes `abca.analytics.read`, `abca.audit.export`, `abca.audit.full_content`
- **CDK:** `AnalyticsRollups`, `ComplianceExportFirehose`, API routes
- **Existing:** `TaskEventsTable`, `emitTaskEvent`, `progress_writer`, Cedar HITL, `approval-metrics-publisher`, `types.ts` / `cli/src/types.ts`

### Alternative solutions

| Alternative | Why not primary |
|-------------|-----------------|
| CloudWatch-only | No stable schema; least-privilege hard |
| OpenSearch for all logs | Cost; access control; duplicates traces |
| Aggregates on `GET /tasks` | Wrong abstraction and auth model |
| Single mega export API | Mixes analytics vs compliance audiences |
| Customer Athena on DDB export only | Still need documented schema + stream delivery |

---

**Note:** Non-triaged RFCs may not get timely review. PRs on non-triaged issues might not be accepted.

* RFC PR:
* Approved by:
* Reviewed by: