[Feat] Explain and stream per-session AI usage across agents, providers, and models

## Overview

Give CAO operators a fine-grained, live explanation of how a session consumes AI-provider resources across supervisors, delegated workers, agent profiles, providers, models, turns, and workflow steps.

This is primarily a **usage explainability** feature, not invoice-grade billing. It should answer where usage occurred, which evidence is authoritative, what CAO contributed to the context, and which portions remain provider-managed or unavailable. Monetary cost can be reported or estimated when defensible, but missing cost must never be represented as zero.

## Problem

A multi-agent CAO session can involve several terminals, providers, models, retries, handoffs, and asynchronous workers. Today an operator cannot answer:

- Which agent or turn consumed the most model resources?
- Which provider and model handled each part of the session?
- How much usage came from the supervisor versus delegated workers?
- How much context did CAO add through agent instructions, skills, memory, security policy, or inter-agent messages?
- Did retries, fallback models, repeated history, or delegation materially increase usage?
- Which numbers are provider-reported, locally measured, estimated, or unavailable?

The existing data model is insufficient for historical attribution:

- terminal metadata persists provider and agent profile, but not the resolved model (`clients/database.py::TerminalModel`);
- the resolved model is passed to the live provider instance during terminal creation but is not retained as durable terminal/session evidence;
- token-related OpenTelemetry semantic-convention names exist, but actual provider usage is not wired into spans or durable records;
- terminal rows are deleted during teardown, including short-lived handoff workers; and
- lifecycle plugin events identify provider/agent but carry no model or usage data.

Provider output is also heterogeneous. Some providers expose input/output/cache tokens, some expose only current context occupancy, some expose provider-native credits, and some expose no usable signal through the interactive CLI. Authentication and billing mode can change the available signal: for example, #237 confirmed that Kiro's `Credits` marker is absent for Q Developer Pro subscription users.

## User Stories

- As a CAO operator, I want a session usage tree so I can identify which agents, providers, models, and turns consumed resources.
- As an agent-profile author, I want to see the estimated contribution of instructions, skills, and recalled memory so I can reduce avoidable context overhead.
- As a workflow author, I want usage correlated with run, step, attempt, and retry so I can compare orchestration strategies.
- As a dashboard client, I want resumable live usage updates followed by final reconciliation so I can show progress without presenting estimates as settled facts.
- As a provider-adapter maintainer, I want to declare exactly which usage signals my provider supports so unsupported measurements remain explicit rather than silently becoming zero.

## Measurement Semantics

The feature must distinguish three commonly conflated concepts:

1. **Cumulative consumption**: tokens or provider-native units processed across model calls. This is the primary usage measure.
2. **Context occupancy**: the current conversation/context-window size. This is a gauge, not cumulative consumption.
3. **CAO-originated payload size**: instructions, memory, skills, messages, and other content known to CAO. This may be processed repeatedly and is usually an attribution estimate rather than a provider billing counter.

A 10k-token context processed five times may represent roughly 50k input tokens even though context occupancy remains 10k. CAO must not substitute occupancy for consumption.

Raw token totals from different models may be displayed and summed descriptively, but must not be presented as equivalent compute, capability, or monetary value.

## Proposed Direction

### 1. Provider capability audit first

Before stabilizing a common schema, document a tested capability matrix for every in-tree provider and relevant authentication/billing modes:

| Capability | Examples |
|---|---|
| Model identity | requested model, actual response model, fallback visibility |
| Token usage | per-turn or cumulative input/output tokens |
| Specialized tokens | cache creation/read, reasoning/thinking |
| Gauges | current context occupancy and context limit |
| Native units | credits, premium requests, weighted usage |
| Monetary data | provider-reported amount/currency, if any |
| Evidence source | structured event, provider API/command, session record, terminal output |
| Stability | CLI versions and redraw/repetition behavior |

For each value, establish what it means, its scope, whether it is a delta/cumulative counter/gauge, and whether it changes by plan or authentication mode. Captured provider fixtures should back the findings where redistribution is safe.

### 2. Generic observation envelope, provider-specific adapters

Use a versioned common envelope while preserving provider-native semantics. Conceptually, each observation needs:

```text
identity
session, terminal, turn, agent profile, provider, requested/response model
optional workflow run, step, attempt, and orchestration relationship

measurement
metric name, value, unit

semantics
scope: model_call | turn | conversation | session | account
behavior: delta | cumulative | gauge
state: provisional | final | corrected

provenance
source: structured_event | native_api | native_command | session_record |
terminal_output | cao_measured | cao_estimate
authority: provider_reported | measured | estimated
provider/adapter version and observation timestamp
```

Normalize common fields only when their meanings match, using OpenTelemetry GenAI vocabulary where applicable:

- input and output tokens;
- cache-creation and cache-read tokens;
- reasoning tokens;
- request count and duration;
- context occupancy and context limit; and
- monetary amount and currency.

Preserve provider-native units such as credits or premium requests alongside common fields. Do not convert them to tokens or currency unless a versioned, documented conversion actually exists.

Each adapter should declare its capabilities and degrade to explicit `unavailable` measurements when evidence is missing or malformed.

### 3. Attribute CAO-originated context

At the point CAO composes or sends content, record contribution categories such as:

- agent/system instructions;
- skill catalog;
- recalled memory;
- security/tool-policy instructions;
- operator message;
- handoff, assign, or send-message payload;
- tool result or workflow input where CAO can observe it.

By default this should retain metadata, counts, tokenizer identity, and optional digests rather than raw prompt/output bodies. Local tokenization must be labelled `measured` or `estimated` according to tokenizer fidelity.

When provider-reported input exceeds the attributable CAO contributions, expose the remainder as **provider-managed/unattributed**. It may include repeated conversation history, native instructions, tool schemas/results, compaction behavior, retries, and other CLI-managed content. Do not fabricate a finer breakdown.

### 4. Durable session usage ledger

Create a durable, append-oriented usage record independent of live terminal rows so usage survives:

- short-lived handoff worker teardown;
- session shutdown;
- server restart; and
- client disconnect/reconnect.

A session summary should project the ledger as:

```text
session
-> terminal / agent instance
-> provider + model
-> turn / dispatch
-> usage observations
-> CAO-attributed contributions
-> provider-managed/unattributed remainder
```

Turn identity should be created when input is dispatched and reconciled when provider evidence or a terminal completion/error boundary arrives. Multiple inputs delivered before a provider exposes a correlatable completion must remain grouped or explicitly ambiguous rather than being assigned arbitrarily.

### 5. Provisional-to-final live stream

Expose a resumable, session-scoped stream, for example:

```text
GET /sessions/{session_name}/usage/stream
Last-Event-ID:
```

The exact endpoint is design work, but the behavioral contract should be:

1. return the current usage snapshot;
2. replay durable events after a monotonic cursor;
3. follow live observations;
4. emit corrected/final revisions as stronger evidence arrives; and
5. emit a final session summary and coverage state.

Representative lifecycle:

```text
usage.turn.started
usage.contribution.recorded
usage.observation.updated # provisional estimate or gauge
usage.turn.reconciled # provider-reported final where available
usage.coverage.changed
usage.session.finalized
```

Updates must carry stable metric identity and revision semantics so a client replaces an earlier estimate instead of adding both estimate and final value. The ledger is committed before publication; the raw terminal event bus is not the source of truth because bounded live queues may drop events under load.

The existing AG-UI stream may carry a secondary `CUSTOM`, `STATE_DELTA`, or metric projection for dashboards. OpenTelemetry may export the same normalized observations. Neither should replace local durable storage or the canonical session usage API.

### 6. Operator presentation

Provide a machine-readable API and a CLI view such as:

```text
cao session usage --tree
cao session usage --follow
```

Example:

```text
Session cao-feature-123 LIVE · 82% measured coverage

code_supervisor · claude_code · claude-sonnet-4-6
Input 84,200 final · Output 9,410 final · Cache read 61,000 final
Turn 1
Agent instructions ~3,200 estimated
Skills catalog ~1,100 estimated
Recalled memory ~900 estimated
Operator message ~250 estimated
Provider-managed 15,850 unattributed

developer · codex · gpt-5
Context occupancy 24,100 provider-reported gauge
Cumulative turn usage unavailable for this provider/version

reviewer · kiro_cli ·
Native credits 0.31 final
```

## Acceptance Criteria

- [ ] A documented, evidence-backed usage capability matrix covers every in-tree provider and known materially different billing/authentication modes.
- [ ] The observation schema is versioned and represents metric, unit, scope, delta/cumulative/gauge behavior, provisional/final state, provenance, and adapter/provider version.
- [ ] Requested provider/model and the actual response/fallback model, where observable, are persisted with terminal/turn identity.
- [ ] Usage can be correlated with session, terminal, agent profile, turn, orchestration relationship, and optional workflow run/step/attempt.
- [ ] Common token dimensions are normalized only when provider semantics match; provider-native measurements remain available without lossy conversion.
- [ ] CAO-originated instructions, skills, memory, messages, and other observable contributions are recorded with explicit measured/estimated provenance.
- [ ] Provider-managed remainder and completely unavailable measurements are shown explicitly; missing values are never rendered as zero.
- [ ] Context occupancy is represented as a gauge and is never substituted for cumulative token consumption.
- [ ] Usage persists after terminal/session teardown and server restart under a documented retention/deletion policy.
- [ ] A session-scoped API returns current aggregate usage and measurement coverage.
- [ ] A resumable live stream supports snapshot, monotonic cursor replay, provisional updates, corrections/finalization, and reconnect without double counting.
- [ ] CLI users can inspect a session tree and follow live usage updates.
- [ ] Adapter tests cover structured and terminal-derived fixtures, malformed/missing evidence, cumulative-to-delta calculation, TUI redraw duplication, model fallback, subscription/no-credit behavior, and provider-version drift.
- [ ] Mixed-provider session tests cover supervisor/worker aggregation, deleted handoff workers, retries, partial coverage, stream reconnect, and final reconciliation.
- [ ] Raw prompt and output bodies are not retained by default solely to support usage attribution.
- [ ] Documentation explains that cross-model token counts are not equivalent compute or cost and distinguishes provider-reported, measured, estimated, unattributed, and unavailable values.

## Suggested Delivery Order

1. Provider capability audit and captured fixtures.
2. Versioned observation/provenance contract.
3. Durable ledger, resolved model identity, and turn correlation.
4. CAO-originated context attribution available across all providers.
5. Initial adapters chosen to demonstrate contrasting semantics: structured tokens, cumulative/gauge-only data, credits, and subscription/unavailable data.
6. Session aggregate API and resumable SSE stream.
7. CLI tree/follow experience.
8. Remaining provider adapters, AG-UI projection, OpenTelemetry export, and optional qualified cost estimates.

This should likely be an umbrella issue with implementation sub-issues for the audit, common ledger/stream, and provider adapters.

## Alternatives Considered

### Lowest-common-denominator token counter

Rejected because it would either discard credits/context/provider-native data or mislabel unlike values as equivalent token consumption.

### Parse every TUI into one fixed schema

Rejected as the primary contract. Screen parsing may be a provider adapter's last-resort evidence source, but formats vary by version, redraws repeat values, and some plans omit fields entirely.

### OpenTelemetry-only implementation

Rejected because local inspection and reconnectable session history should not depend on an external collector. OTel is an export surface over the same observations.

### Billing-export reconciliation

Useful as a future adapter, but not required for the core goal. Provider billing exports may be delayed or impossible to correlate to an individual CAO terminal/turn.

## Non-Goals

- Guaranteeing invoice-grade cost for every provider and authentication mode.
- Treating absent cost as zero or subscription usage as free compute.
- Inferring provider-internal tool calls, retries, history assembly, or reasoning breakdown without evidence.
- Claiming raw token totals are directly comparable across models/providers.
- Retaining full prompts or model outputs by default.

## Risks and Open Questions

- Which providers expose stable machine-readable usage while operating in CAO's interactive mode?
- Can provider session records be consumed safely without retaining or exposing conversation content?
- How should overlapping/eager input delivery be represented when a provider offers only conversation-level cumulative counters?
- Which tokenizer/version should estimate each model's CAO-originated contributions, and how should unknown models degrade?
- Should usage retention follow workflow-journal policy or have an independent bound?
- Which usage metadata belongs on the fleet-wide AG-UI stream versus the session-scoped stream?

The strongest risk is false precision. The feature is valuable only if it remains honest about semantic differences and incomplete observability while still providing the best available attribution.

Start with the provider capability audit, then read clients/database.py::TerminalModel and the existing AG-UI stream mentioned in the proposal. Review the provider and authentication-mode evidence before settling the versioned observation schema and durable ledger boundaries. Done means the acceptance criteria are met: persisted, provenance-aware usage, explicit unavailable values, resumable updates, and CLI session views.

Written by the indexing model from the issue text.