Tracking: ADK 2.0 — workflow- and team-aware tracing across producer and consumer

# Tracking: ADK 2.0 — workflow- and team-aware tracing across producer and consumer

> **Revision history**
> - v2: applied review corrections — fixed schema column count, clarified that `node_info.run_id` / `parent_run_id` are computed properties, called out empty-path semantics, tightened `AGENT_TRANSFER` and `EVENT_COMPACTION` payload contracts, made `TOOL_PAUSED` pairing explicit, replaced the unreliable `invoke_node` duration acceptance criterion, marked workflow node boundary events as design-required (no `BasePlugin` callback exists), corrected SDK file paths to the current `src/bigquery_agent_analytics/...` layout, added four sub-issues (schema_version, route/widgets/rewind, null-safe pre-2.0 tests, workflow fixture).
> - v3: precision pass — added the producer prerequisite that `EventData` does not currently carry the source `Event` (so node-info enrichment is only feasible for rows with an originating ADK Event, or after plumbing changes); replaced the raw-mirror of internal `isolation_scope` with a producer-owned normalized `attributes.adk.scope = {id, kind}`; added an `attributes.adk.pause_kind` discriminator on `TOOL_PAUSED` so HITL synthetic pauses (`adk_request_*`) don't pollute tool-duration analytics; corrected `AGENT_STATE_CHECKPOINT` trigger to fire on either `agent_state` or `end_of_agent` independently (ADK can emit either without the other); demoted `attributes.adk.otel_span_id` to best-effort and made `attributes.adk.source_event_id` the reliable join key, since ADK records associated event IDs on spans rather than the reverse.
> - v4: cleanup pass — split the acceptance contract so `attributes.adk.schema_version` applies to every enriched row but `attributes.adk.source_event_id` only to rows with an originating ADK `Event`; picked `attributes.adk.scope = null` as the single canonical unscoped representation; widened `scope.kind` to `{function_call, node_run, root, unknown}` and required fixture coverage for full-path node scopes, function-call-id scopes, and unscoped rows; renamed the last stale "isolation-scope cardinality view" reference; replaced the `periodic_materialization` terminology (which is only an `examples/migration_v5/` artifact) with the actual SDK patterns `materialize_window.py` / `ontology_materializer.py`.
> - v5: precision pass — fixed a real bug in v4's `scope.kind` derivation (the rule misclassified bare `name@run_id` as `function_call` because the order-of-evaluation wasn't specified — now node-path/`name@run_id` shapes are checked **before** function-call IDs); dropped `root` from `scope.kind` (current ADK has no explicit root marker, so `scope = null` is the only unscoped form and a `root` kind would never fire); added a producer requirement to pair `TOOL_PAUSED` → `TOOL_COMPLETED` from the **event / user-message path** (not only `after_tool_callback`), since user-supplied long-running tool responses arrive there and the v4 contract was too optimistic for cross-invocation pauses; fixed the `workflow_invocations` view source phrasing so it doesn't contradict the Event-originating split; corrected the materialization wording to reference `ViewManager` / CLI as the actual SDK view-deployment surface, with `materialize_window.py` reserved as a pattern source if a dedicated analytical-table materializer is later needed.
> - v6: implementation-risk pass — replaced `parent_run_id` as the DAG join key with a producer-serialized `attributes.adk.node.parent_path` (run ids are not unique across nodes/branches, so the v5 contract would have produced corrupt parent/child joins); specified the `TOOL_PAUSED` pause registry contract — composite key `(session_id, function_call_id)`, reconstructable from prior `TOOL_PAUSED` rows rather than process-local memory, with explicit dedupe semantics for `TOOL_COMPLETED`; called out `TOOL_PAUSED` as a sixth new event type and added consumer sub-issues to update `EventType` in `trace.py` and `event_semantics.py` (not only `_EVENT_VIEW_DEFS`); scoped the first `branch_fanout` view to cardinality only and deferred the concurrency dimension behind the workflow-node-boundary design decision; added a dedicated `AGENT_STATE_CHECKPOINT` offload contract sub-issue (`HybridContentParser` is content-part-shaped, not state-shaped, so state snapshots need their own contract for URI/SHA/inline-removal). [v7 later widened this pause key — see below.]
> - v7: consistency drift pass — propagated `parent_path` through the consumer-gaps section and acceptance criteria (both were still mentioning `parent_run_id`); defined the canonical root `parent_path = null` so bare paths like `name@1` (no slash) get one unambiguous output; widened the pause registry key from `(session_id, function_call_id)` to `(app_name, user_id, session_id, function_call_id)` so a shared BigQuery table aggregating multiple apps/users can't produce cross-tenant collisions; rewrote the back-compat risk for oversized agent-state snapshots to point at the new `AGENT_STATE_CHECKPOINT` offload design sub-issue (the v5 "reuse `HybridContentParser`" wording survived into v6 and contradicted the v6 demotion).
> - v8: identity-propagation pass — added `attributes.adk.app_name` (sourced from `InvocationContext.app_name`) on every ADK-enriched row, since the existing 16-column schema has no top-level `app_name` and the v7 pause-registry key would otherwise have nowhere to read it from on the consumer side; updated the `long_running_tool_durations` view pairing and the consumer dedupe SQL to join on the full `(app_name, user_id, session_id, function_call_id)` tuple (v7 widened the contract but left the consumer-side SQL on the old partial key, reintroducing the collision the contract was meant to prevent); applied the same identity rule to `workflow_invocations` and stated it as a general rule for every cross-event ADK 2.0 view; fixed a `replace_all` typo in the v7 history entry (it claimed the key was widened from the new tuple to itself).
> - v9: cleanup pass — applied the full-key grouping rule explicitly to every cross-event view bullet (`agent_transfer_chains`, `branch_fanout`, `compaction_windows`, `scope_cardinality`) so v8's preamble can't drift; updated the acceptance criterion so `TOOL_PAUSED` / `TOOL_COMPLETED` match on the full telemetry key, not just `function_call_id`; broadened the SDK event-type-registration sub-issue from `trace.py` + `event_semantics.py` to "audit every event-type allowlist / categorization surface" (including `trace_evaluator.py::_DEFAULT_EVENT_TYPES` and UDF label maps); added a BigQuery-streaming-visibility design note to the pause registry (immediate-after-insert queries can return false `pause_orphan` flags due to eventual visibility — sub-issue covers either delayed reconstruction or supplementing from in-session ADK history); dropped the stale "v6 contract" label from the producer checklist.
> - v10: cleanup pass — reconciled an internal contradiction in the pause-registry contract (v6 said "not a process-local in-memory map" but v9 added an in-process-cache option; v10 names BigQuery + ADK session history as the durable sources of truth, with any process cache positioned as a latency-optimization layer only); fixed a self-inflicted v7 `replace_all` artifact in the v6 history that had retroactively rewritten v6's original `(session_id, function_call_id)` pause key as the v7 wider tuple; expanded the SDK event-type acceptance criterion to require auditing the full SDK type surface (not only `EventType` + `event_semantics.py`); added "using full telemetry identity" to every consumer view checklist item so the rule can't be lost when sub-issues are filed; defined explicit `scope = null` behavior in `scope_cardinality` (emit a separate `unscoped_count` column rather than letting nulls vanish through `COUNT(DISTINCT)`).
> - v11: precision pass — added the full-telemetry-identity wording to the `workflow_invocations` checklist item (the v10 sweep missed this one); rewrote the pause-registry producer checklist item to match the v10 layered contract (durable BQ + ADK history with process cache as optional latency layer, and `pause_orphan` gated on the chosen strategy failing); resolved the `scope_cardinality` output-grain ambiguity (one row per kind including a synthetic `kind = 'unscoped'`, replacing the v10 per-row `unscoped_count` column — that approach repeated the unscoped count on every kind row); removed the brittle prefix-based `function_call` classification rule (ADK only generates `adk-` prefixes when an ID is missing; model-provided IDs can be `call_xyz`, `toolu_abc`, etc., so v11 makes `function_call` the default for any non-node-run non-empty string, with `unknown` reserved for empty or non-string anomalies); added a deployment-surface sub-issue and acceptance for the cross-event analytical views (current `ViewManager` and CLI are per-event-type only in `views.py:281` / `cli.py:1659`, so the six new views need their own deployment plumbing — extension or sibling manager).
> - v12: cleanup pass — synced the producer scope-derivation checklist item to the v11 rule (was still carrying the v10 "known function-call ID shape" wording that would have reintroduced the prefix-brittleness bug at sub-issue filing time); synced the consumer `scope_cardinality` checklist item to row-per-kind grain (was still mentioning the discarded `unscoped_count` column); separated true v2 unscoped from pre-2.0 contamination in `scope_cardinality` (`COALESCE(...kind, 'unscoped')` would have bucketed legacy rows with no `attributes.adk` block as `unscoped`; v12 gates the view on `attributes.adk.schema_version IS NOT NULL` and adds a null-safe test); promoted cross-event view deployment from sub-issue-only to a top-level acceptance bullet (otherwise the tracker could pass with SQL that no public command deploys); defined `pause_orphan` semantics in `long_running_tool_durations` (orphan completions now surface as `status = 'orphan_completion'` rows with null `pause_ts`, so the producer's drift-quantification contract actually appears in the primary duration view).
> - v13: precision pass — narrowed the `scope_cardinality` gate from `schema_version IS NOT NULL` (which lets in non-Event-originating callback rows like `USER_MESSAGE_RECEIVED`, before-model, before-tool — they're ADK-enriched but have no meaningful `Event.isolation_scope`) to `source_event_id IS NOT NULL` so only rows that actually originated from an ADK `Event` enter the bucketing; split the `scope_cardinality` value column into two — `distinct_scope_count` and `event_count` — so the "distinct scope IDs" metric for scoped rows and the "event volume" metric for unscoped rows stop sharing one overloaded column; extended the `pause_kind` contract onto `TOOL_COMPLETED` rows emitted from the long-running event/user-message path (orphan completions otherwise have no `TOOL_PAUSED` partner to derive `pause_kind` from, so HITL completion orphans could mix into tool-duration drift), and required `pause_kind = 'tool'` on orphan rows included in `long_running_tool_durations`; added the orphan-row/status requirement to the `long_running_tool_durations` consumer checklist item so the filed sub-issue doesn't regress to a pure inner-join duration view.
> - v14: correctness pass — renamed v13's `scope_cardinality.event_count = COUNT(*)` to `source_event_count = COUNT(DISTINCT source_event_id)` because one ADK `Event` already fans out into multiple BQAA rows on `on_event_callback` (`STATE_DELTA`, HITL, A2A, `AGENT_RESPONSE` can all come from the same source Event at `:2991`, `:3001`, `:3067`, `:3109`), so `COUNT(*)` was counting BQAA rows, not distinct Events — the metric label was lying about its units; kept `row_count = COUNT(*)` as a diagnostic-only column; propagated the v13 `TOOL_COMPLETED.pause_kind` requirement to the acceptance criterion and the producer checklist (both were still talking about `TOOL_PAUSED` only — the proposal section had it but the contract surfaces filed sub-issues read from did not); added a HITL-orphan acceptance test that exercises an orphan `TOOL_COMPLETED` with `pause_kind != 'tool'` and asserts it's excluded from `long_running_tool_durations`.
> - v15: precision pass — reconciled an internal contradiction in the orphan acceptance bullets (one said paired+orphan filtered to `pause_kind = 'tool'`, the next said "every `TOOL_COMPLETED` with `pause_orphan = true`" — the second was too broad and would have included HITL orphans the first was meant to exclude); specified the ADK-float-epoch-seconds to BigQuery `TIMESTAMP` conversion for `compaction_windows` (ADK `Event.timestamp` and `EventCompaction.start_timestamp` / `end_timestamp` are float seconds per `events/event.py:156` and `event_actions.py:42-45`, while BQAA top-level `timestamp` is a BQ `TIMESTAMP`; without an explicit microsecond-preserving conversion the view would silently truncate fractional windows); added an explicit `WHERE NULLIF(...node.path, '') IS NOT NULL` filter to `workflow_invocations` so Event-originating non-workflow rows (`NodeInfo.path` defaults to `""` per `events/event.py:45`) don't produce a meaningless null-node DAG row — `NULLIF` alone made the read null-safe but didn't keep the empty/non-workflow rows out of the grouping.
> - v16: umbrella ↔ sub-issue alignment — restated the HITL-orphan acceptance bullet to match #199's clarified producer contract (HITL completions stay on the `HITL_*_COMPLETED` flow; the HITL-orphan `TOOL_COMPLETED` is a **synthetic** consumer-fixture row, not a producer output). The v14 wording "produced by the fixture" could be read literally as "make the plugin emit it", which contradicts #199. Also propagated the same correction to the sub-issue surfaces: #194 broadened its telemetry-context-struct minimum field set so #195/#197/#198/#199/#200/#201/#202/#203/#205 are all implementable; #199 spelled out the id→name derivation against `_HITL_EVENT_MAP` (key the lookup on `part.function_call.name`, not the id); #215 specified separate-stream dedupe (single-partition `ROW_NUMBER` would have dropped completion rows under duplicate paused rows); #218 moved the synthetic-fixture-row provenance from cross-issue comments into the issue body; #220 added #203 and #208 as blockers because both can add documented fields.
> - v17: umbrella catch-up — backported four contract changes from the sub-issues into the umbrella body so #190 stops being a stale source of guidance: producer prerequisite now says "source `Event` or a typed telemetry context struct" (not "pre-extracted node-info struct" — the v15 wording was too narrow per #194); producer `TOOL_PAUSED` bullet now carries #199's `_HITL_EVENT_MAP` id→name derivation rule; consumer `branch_fanout` bullet now has #214's `source_event_id IS NOT NULL` gate and zero-branch policy; consumer `long_running_tool_durations` bullet now mirrors #215's healthy-vs-orphan split with the explicit `pause_orphan IS NOT TRUE` exclusion on the healthy-pair completed stream (without which a `TOOL_COMPLETED` with `pause_orphan = true` could appear both as a healthy pair AND as an orphan when the paused row becomes visible later). Also tightened #211 to list `_EVENT_VIEW_DEFS` in the audit set and to require the existing `TOOL_COMPLETED` view to surface the new long-running ADK fields (or to document that consumers must read them from `long_running_tool_durations` / base-table JSON).
> - v18: checklist catch-up — v17 fixed the prose sections but the **checklist items** below were still pre-v17. v18 propagates the same four contracts down into the checklists implementers copy from: producer `TOOL_PAUSED` checklist now spells out the #199 id→name derivation; consumer event-type-audit checklist now lists `_EVENT_VIEW_DEFS` explicitly and references the #211 `TOOL_COMPLETED` shape decision; consumer `branch_fanout` checklist now states the `source_event_id IS NOT NULL` gate and zero-branch row policy; consumer `long_running_tool_durations` checklist now states the `pause_orphan IS NOT TRUE` healthy-stream exclusion and per-stream dedupe. Also fixed sub-issue cross-references: #211 carves out node-boundary typed views as blocked by #207 (the payload shape is not yet designed); #213, #214, #216 add #195 (`app_name` identity stamping) as an explicit blocker since each view groups on `JSON_VALUE(attributes, '$.adk.app_name')`; #221 docs sub-issue updates the placeholder `C2-C7`/`#C10` references to the actual filed numbers (#212-#217 and #220), bumps the tracker reference to v17, and adds recipe-content requirements for the zero-branch policy and the healthy/orphan split.
> - v19: acceptance catch-up + sub-issue corrections — v18 caught up the checklist but the **Acceptance criteria** section was still pre-v17. v19 propagates v17/v18 into the acceptance bullets: event-type surface acceptance now lists `_EVENT_VIEW_DEFS` explicitly with the #211/#207 node-boundary carve-out and the #211 `TOOL_COMPLETED` shape decision; long-running-tool acceptance now requires the `pause_orphan IS NOT TRUE` exclusion on the healthy-pair stream (orphan rows only in the orphan branch). Also fixed a real correctness slip in #211 (`TOOL_PAUSED` typed view was listed as including `attributes.adk.pause_orphan`, but `pause_orphan` is a `TOOL_COMPLETED`-only field per #199 — including it would produce an always-null column and blur the producer contract). Added missing #195 blockers to #212 and #215 (both group on `JSON_VALUE(attributes, '$.adk.app_name')` directly, so the dependency is more than transitive). Added #211 as a blocker on #220 (the existing-`TOOL_COMPLETED`-view shape decision lands in `SDK.md`). Bumped stale v15 tracker references on #212 and #220 to v18.
> - v20: HITL-orphan acceptance wording — v19's "measure those from the typed `TOOL_COMPLETED` / `HITL_*` views instead" conflicted with three earlier decisions: per #199 the producer doesn't emit HITL completions as `TOOL_COMPLETED` (they stay on `HITL_*_COMPLETED`); per #218 the HITL-orphan `TOOL_COMPLETED` row is synthetic defensive fixture data, not a producer output or metric surface; per #211 the existing `TOOL_COMPLETED` typed view may or may not expose long-running ADK fields. Replaced with: HITL waits/completions are excluded from `long_running_tool_durations`, real HITL behavior is analyzed from the typed `HITL_*` event views, and the synthetic HITL-orphan `TOOL_COMPLETED` fixture row is defensive coverage only, not a metric source.

## Context

ADK 2.0 introduced a workflow runtime, the Task API, event compaction, agent-state checkpointing, and long-running tool semantics. These show up on the `Event` stream as new fields (`node_info`, `actions.compaction`, `actions.agent_state`, `actions.end_of_agent`, `actions.transfer_to_agent`, `isolation_scope`, `branch`, `long_running_tool_ids`, and `actions.route` / `actions.render_ui_widgets` / `actions.rewind_before_invocation_id`). The BQAA producer (the in-repo plugin shipped with ADK at `src/google/adk/plugins/bigquery_agent_analytics_plugin.py`) and the BQAA consumer (this SDK, primarily `src/bigquery_agent_analytics/views.py`) were both designed against the 1.x flat-callback model and only partially reflect the new surface today. This issue is the umbrella for closing that gap end-to-end.

Audience for the resulting telemetry: anyone running ADK 2.0 multi-agent teams (sequential/parallel/loop workflows, A2A delegation, Task API sub-agents) who needs to answer "where in the workflow tree did this happen", "which agent handed off to which", "how long was this tool actually running across pauses", and "what was the agent state at this checkpoint".

## What's new in ADK 2.0 that affects observability

Confirmed against `adk-python@main`:

- **`Event.node_info: NodeInfo`** (`src/google/adk/events/event.py:35`) carries `path` (e.g. `wf/A@1/B@2`, defaulting to `""` for non-workflow events), `output_for` (ancestor paths this event satisfies), and `message_as_output`. **`run_id` and `parent_run_id` are computed @property values derived from `path`** via `_NodePathBuilder`, not stored fields — the producer must serialize them explicitly under `attributes.adk.node`.
- **`Event.branch`** — multi-agent silo path (e.g. `agent_1.agent_2`). The in-repo plugin already captures this for `AGENT_RESPONSE` only (line 3118).
- **`Event.isolation_scope`** — Task API scope tag (e.g. `` or `@`) used to scope session visibility for delegated sub-agents. Not captured anywhere today. **Note:** both `Event.isolation_scope` and `InvocationContext.isolation_scope` are marked internal in the ADK source — semantics may change. The producer must normalize (see proposal §2) rather than mirror the raw value as a stable public contract.
- **`EventActions.compaction: EventCompaction`** — `start_timestamp`, `end_timestamp`, `compacted_content`. Marks where the runtime collapsed event ranges. (Note: there is no explicit "compacted_count" field; consumers derive affected row counts from the timestamp window.)
- **`EventActions.transfer_to_agent: str | None`** — the target agent name only. The plugin recognizes `TransferToAgentTool` in `tool_origin` (line 227), but does not emit a first-class transfer event.
- **`EventActions.end_of_agent: bool`** + **`EventActions.agent_state: dict`** — resumable agent boundary + state snapshot. Not captured.
- **`EventActions.route`**, **`render_ui_widgets`**, **`rewind_before_invocation_id`** — workflow routing value, UI-widget side effects, and invocation-rewind requests. None captured.
- **`Event.long_running_tool_ids: set[str]`** — async-tool markers (function-call IDs). Used as a guard in the plugin (line 3092) but never surfaced as data. **Note:** this set also contains the synthetic `adk_request_credential` / `adk_request_confirmation` / `adk_request_input` IDs that ADK injects for HITL waits — see `_HITL_EVENT_MAP` at line 87. The producer must discriminate so HITL pauses do not contaminate tool-duration analytics.
- **OTel spans** (`src/google/adk/telemetry/`): `invoke_agent`, `invoke_workflow`, `invoke_node` with semconv v1.36/v1.41 attributes. The plugin's `TraceManager` integrates with OTel, but workflow spans aren't yet correlated with BQAA rows on the consumer side.

## Producer gaps (in-repo plugin)

`bigquery_agent_analytics_plugin.py` overrides 12 plugin callbacks. After auditing each path against the 2.0 event model:

| 2.0 surface | Status | Where |
|---|---|---|
| `event.branch` | Logged (AGENT_RESPONSE only) | line 3118 |
| `event.long_running_tool_ids` | Used as guard only, not surfaced | line 3092 |
| `event.node_info` (path / derived run_id, parent_run_id) | **Not captured** | n/a |
| `event.isolation_scope` | **Not captured** | n/a |
| `actions.compaction` | **Not captured** | n/a |
| `actions.transfer_to_agent` | Tool-origin tagged, but no transfer event | line 227 |
| `actions.end_of_agent` / `actions.agent_state` | **Not captured** | n/a |
| `actions.route` / `render_ui_widgets` / `rewind_before_invocation_id` | **Not captured** | n/a |
| Workflow agents (Loop/Parallel/Sequential) | No iteration/branch boundary events | n/a |

The single point where most of these arrive is `on_event_callback` (line 2956), which today only branches on `state_delta`, HITL function calls, A2A metadata, and final-response guards. Everything else flows through unobserved.

## Consumer gaps (this SDK)

The existing top-level `agent_events` schema (16 columns: `timestamp`, `event_type`, `agent`, `session_id`, `invocation_id`, `user_id`, `trace_id`, `span_id`, `parent_span_id`, `content`, `content_parts`, `attributes`, `latency_ms`, `status`, `error_message`, `is_truncated`) is intentionally stable. The producer can carry all the ADK 2.0 fields under `attributes` without a column-level migration, but `src/bigquery_agent_analytics/views.py` (`_EVENT_VIEW_DEFS`) currently exposes generic per-event-type views only. There is no:

- Workflow-tree view that reconstructs `node_info.path` into a queryable DAG (parent/child via the producer-serialized `attributes.adk.node.parent_path`; see proposal §1 for why `parent_run_id` alone is unsafe).
- Agent-transfer chain materialization (multi-agent team handoffs are not first-class).
- Compaction-window view (so `LLM_REQUEST`/`LLM_RESPONSE` cardinality is misleading in long sessions).
- Long-running-tool duration metric that accounts for pauses (pause may cross invocation boundaries — `invoke_node` span duration is not a reliable proxy).
- Branch-fanout analytics (parallel agent silos).
- Scope-cardinality view (how wide is Task API branching, grouped by the producer-normalized `attributes.adk.scope.kind`).

This is the analytics surface design partners will reach for once we start producing the data.

## Proposal — producer changes (in-repo plugin)

Schema strategy: **no new top-level columns**. All ADK-2.0-specific data lands under `attributes.adk.*` so it ships immediately, stays back-compat with pre-2.0 rows, and leaves room for a future v6-style consolidation if the access pattern justifies it.

**Prerequisite (producer plumbing).** Today, `EventData` (line 1949) is a typed container of span/latency/model/usage fields only — it does **not** carry the source `Event`. Only `on_event_callback` has the originating `Event`; the LLM and tool lifecycle callbacks (`before_model`, `after_model`, `before_tool`, `after_tool`, etc.) operate on `CallbackContext` + `LlmRequest` / `LlmResponse` / tool args. As a result, node/branch/scope enrichment cannot be applied uniformly across "every row" without first plumbing **either the source `Event` reference itself, or a typed telemetry context struct carrying at minimum `event_id`, `node_info`, `branch`, `isolation_scope`, `actions`, `long_running_tool_ids`, `content`, `timestamp`, and `custom_metadata`** (per #194 — a literal `NodeInfo + EventActions` is too narrow because #195/#197/#198/#199/#200/#201/#202/#203/#205 each need different event-derived fields). The first producer sub-issue is this plumbing. Until it lands, the contract is: **node/branch/scope enrichment is best-effort and may be null on rows that don't have an originating `Event`** (e.g., before-model / before-tool rows that fire before the runner produces an event).

1. **Enrich rows that have an originating `Event` with `attributes.adk.node`** in `_log_event` (line 2815) after the plumbing above:
- `node.path` (raw, may be `""`), the path-derived `node.run_id`, and **`node.parent_path`** — the canonical DAG join key, **not** `parent_run_id` (run ids are only unique within a single segment, so two different parent nodes can share `parent_run_id` and joining on it alone would corrupt the DAG).
- **Canonical `parent_path` computation:** if `node.path` contains a `/`, `parent_path` is everything before the final `/@`. **If `node.path` is bare (no `/`, e.g. `name@1`) or empty, `parent_path = null`** — this is the single canonical "no parent / root" form. Consumer views use `NULLIF(JSON_VALUE(attributes, '$.adk.node.parent_path'), '')` defensively, but the producer should emit JSON `null`, not `""`.
- `node.parent_run_id` may stay as a serialized convenience field for debugging / ordering within a single parent, but consumer views must join on `parent_path`.
- Also `node.output_for`, `node.message_as_output`.
2. **Enrich originating-Event rows with `attributes.adk.branch`** and a producer-owned, normalized **`attributes.adk.scope`** in place of mirroring the raw `Event.isolation_scope`. The contract:
- `attributes.adk.scope = null` for unscoped rows. This is the **single canonical** unscoped form. There is no `root` kind — current ADK has no explicit root marker (see `workflow/_workflow.py::_compute_isolation_scope_for_node`), it just emits `None` for unscoped/root-ish cases.
- For scoped rows, `attributes.adk.scope = {id, kind}`:
- `scope.id`: stable identifier for the scope (the raw `isolation_scope` string is a reasonable seed but the contract is producer-owned).
- `scope.kind`: one of `function_call`, `node_run`, **`unknown`**.
- **Derivation order matters — evaluate in this sequence, return on first match:**
1. Raw value is `None` → `scope = null` (do not emit `{id, kind}` at all).
2. **Node-run shape**: value matches `name@` (bare) or `/@` (full path emitted by task-mode workflows, per `workflow/_workflow.py::_compute_isolation_scope_for_node`) → `kind = node_run`.
3. **Function-call default**: any non-empty string that did not match step 2 → `kind = function_call`. **Do not depend on prefix shape** (`adk-...`, `call_...`, `toolu_...`, etc.) — ADK only generates `adk-` prefixes when an ID is missing in `flows/llm_flows/functions.py:180`, and model-provided IDs from the FC delegation path (`workflow/_llm_agent_wrapper.py:135`) can use any provider-native format. The classification works because by step 3 we've already eliminated node-run shapes; anything else is an opaque ID that ADK is using as a scope handle.
4. Empty string, non-string, or other unmistakably-malformed value → `kind = unknown` with a producer-side warning log. **Never silently coerce into a known kind for genuinely anomalous input** — `unknown` is the visibility signal for ADK scope-shape drift, but it should not fire on legitimate model-provided function-call IDs.
- Two earlier-version bugs locked in by step ordering: v4's "no path separator → function_call" misclassified bare `name@run_id` (step 2 now fires first); v10's "known/generated function-call shape" implicitly required an `adk-` prefix or similar (step 3 now classifies by what it isn't, not by what it looks like).
- **Fixture requirement (consumer side too):** the representative workflow fixture must cover (a) full-path node scopes like `/@`, (b) **bare** node scopes like `name@` (the same `node_run` kind but without a path separator — the case v4 misclassified), (c) **at least two** function-call-id scopes — one ADK-generated `adk-...` and one **model-provided non-`adk-` ID** like `call_abc123` or `toolu_xyz` (the case v10 misclassified by relying on prefix shape), and (d) unscoped rows. Tests assert the correct `scope.kind` (or `scope = null`) for each.
- Document explicitly in `SDK.md` that raw ADK `isolation_scope` is **not** a stable public API and consumers should join on `attributes.adk.scope`, not on the raw value.
3. **Stamp `attributes.adk.schema_version`** on every row touched by the ADK enrichment path (single source of truth for consumer gating). Stamp **`attributes.adk.app_name`** (from `InvocationContext.app_name`) on every ADK-enriched row as well — the existing 16-column schema has no top-level `app_name`, so without this the pause-registry key and the cross-event view identity rules below cannot be evaluated on the consumer side. (Don't promote it to a top-level column — schema strategy says no new columns.)
4. **Stamp `attributes.adk.source_event_id`** on every row that originates from an `Event` — this is the reliable join key against ADK's OTel `associated_event_ids` span attribute (which records event IDs on spans, not span IDs on events).
5. **Emit new event types** in `on_event_callback`:
- `AGENT_TRANSFER` from `event.actions.transfer_to_agent` (the ADK field is just the target agent name string). Required content: `{from_agent, to_agent}`. `reason` is **optional** — include only when it can be inferred from a co-occurring tool result or message; never invent.
- `EVENT_COMPACTION` from `event.actions.compaction`. Required content: `{start_timestamp, end_timestamp, compacted_content}` (the three fields ADK actually exposes). Affected row counts are derivable from the timestamp window on the consumer; do not require the producer to compute a count.
- `AGENT_STATE_CHECKPOINT` — **trigger fires when either `event.actions.agent_state is not None` or `event.actions.end_of_agent is True`**, independently. ADK's `_create_agent_state_event` can produce either field on its own (e.g. `{agent_state: null, end_of_agent: true}` when the workflow clears the snapshot at end-of-agent), and the docstring on `EventActions.end_of_agent` notes there can be multiple `end_of_agent=True` events for the same agent within a loop. Content shape: `{end_of_agent: bool, agent_state: dict|null}` when small enough to inline; **the offload contract for oversized snapshots is a separate sub-issue** (see producer checklist) — `HybridContentParser` is content-part / text-shaped, and a structured-dict snapshot needs its own explicit URI/SHA/inline-removal contract.
- `WORKFLOW_NODE_STARTING` / `WORKFLOW_NODE_COMPLETED`. **Design-required**: `BasePlugin` exposes no node lifecycle callback, so these must be derived either (a) by consuming OTel `invoke_workflow` / `invoke_node` span start/end, or (b) deterministically from the first and last event observed per `attributes.adk.node.path` within an invocation. Pick one in a follow-up sub-issue; option (a) is more accurate, option (b) is simpler and OTel-independent.
6. **Surface long-running tools with an explicit pairing contract + pause-kind discriminator**:
- `event.long_running_tool_ids` is a set of function-call IDs. When non-empty, emit `TOOL_PAUSED` once per id, with `attributes.adk.function_call_id` (== the long-running id) on the row.
- Required discriminator `attributes.adk.pause_kind ∈ {tool, hitl_credential, hitl_confirmation, hitl_input}` on **both `TOOL_PAUSED` and `TOOL_COMPLETED`** rows emitted from the long-running event/user-message path. **Derivation (per #199 — id→name lookup, not id→map):** for each `long_running_tool_id`, locate `part.function_call.id == long_running_tool_id` in the source event's `content.parts`, then look up `_HITL_EVENT_MAP[part.function_call.name]` (line 87) — `adk_request_credential` / `adk_request_confirmation` / `adk_request_input` → matching `hitl_*` kind; otherwise `tool`. **Do not look up `_HITL_EVENT_MAP` against the id value itself** — that would classify every HITL pause as `'tool'` and defeat the discriminator. If no matching `part.function_call` is found in the event (malformed), emit `pause_kind = 'tool'` and log a warning. Stamping `pause_kind` on `TOOL_COMPLETED` too is required because orphan completions have no matching `TOOL_PAUSED` to inherit the kind from. The consumer's `long_running_tool_durations` view filters to `pause_kind = 'tool'` on both paired rows and orphan rows so HITL waits never inflate tool-duration analytics. (HITL waits already get their own typed events via the existing `HITL_*` flow; the discriminator just prevents double-counting.)
- **Pair `TOOL_PAUSED` → `TOOL_COMPLETED` from both the tool-callback path AND the event / user-message path.** `after_tool_callback` only fires for normal in-process tool execution; user-supplied long-running tool responses (the actual cross-invocation case) arrive as `function_response` parts on user messages or events and never go through `after_tool_callback`. The existing plugin already inspects `function_response` parts in `on_event_callback` (`bigquery_agent_analytics_plugin.py:3022`) and `on_user_message_callback` for HITL detection — extend the same iteration to detect **non-HITL** `function_response.id` values and emit `TOOL_COMPLETED` with `attributes.adk.function_call_id` = `function_response.id`.
- **Pause registry contract (must be specified before implementation):**
- **Key**: composite `(app_name, user_id, session_id, function_call_id)`. The full ADK session identity is needed because the BigQuery `agent_events` table can aggregate multiple apps and users — `(session_id, function_call_id)` alone can collide cross-tenant. `app_name` comes from `InvocationContext.app_name`; `user_id` and `session_id` are already row columns.
- **Durable source of truth: BigQuery `TOOL_PAUSED` rows + ADK in-session history.** On the completion path the producer looks up unmatched pauses by `(app_name, user_id, session_id, function_call_id)`. A process-local map is **not** the durable source — it survives only as a latency-optimization cache layered in front of the durable sources (see the design sub-issue below). This separation is what survives process restarts, multi-worker routing, and stale memory.
- **BigQuery streaming visibility caveat (design sub-issue):** rows written via the Storage Write API have eventual visibility — a `function_response` arriving seconds after the matching `TOOL_PAUSED` row was streamed can read back zero matches if BigQuery is the only consulted source. The pause-registry sub-issue must resolve this with one of: (a) an in-process cache as a latency optimization in front of BigQuery as the durable fallback, (b) reconstruction from in-session ADK history events that carry `long_running_tool_ids` when available, or (c) a settling-time grace window before declaring orphans. Pick one; do not silently rely on immediate read-after-write.
- **`pause_orphan` gate:** `pause_orphan = true` is only emitted **after** the chosen strategy fails to find a match — i.e., not on an immediate BQ miss alone. The sub-issue's acceptance test must demonstrate this gating (stream a `TOOL_PAUSED`, then immediately observe a matching `function_response`, and verify the row is not marked orphan).
- **TTL**: rows older than `BQAA_PAUSE_REGISTRY_TTL_HOURS` (default 24h) are considered abandoned. The view-side `long_running_tool_durations` excludes them or surfaces them as `status = 'abandoned'`.
- **Dedupe**: `TOOL_COMPLETED` for a given `(app_name, user_id, session_id, function_call_id)` is emitted **at most once**. The producer guards against duplicate observation (the same `function_response` arriving via both an event and a user message) by checking the registry before emit. The consumer further dedupes via `ROW_NUMBER() OVER (PARTITION BY JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, JSON_VALUE(attributes, '$.adk.function_call_id') ORDER BY timestamp)` — the full key, not the old partial.
- **Missed pauses**: if a `function_response` is observed without a matching prior `TOOL_PAUSED`, the producer logs a warning and emits the `TOOL_COMPLETED` with `attributes.adk.pause_orphan = true` so analytics can quantify drift.
7. **Stop using `on_state_change_callback`** (deprecated, line 3125) — delete the stub and the warning path.
8. **OTel correlation (best-effort)**: copy the current OTel span's `span_id` into `attributes.adk.otel_span_id` **only when it can be shown to reliably point at the workflow node span**. ADK records `associated_event_ids` on its node spans, not span IDs on events, so the reliable Cloud-Trace join is `OTel.associated_event_ids ↔ attributes.adk.source_event_id`. `otel_span_id` is a convenience field whose accuracy is callback-dependent; document this in `SDK.md`. The full reliability story belongs to a future OTel exporter path.

These changes funnel through `on_event_callback` and `_log_event`, so the diff is concentrated rather than spread across every callback.

## Proposal — consumer changes (this SDK)

New view definitions belong in `src/bigquery_agent_analytics/views.py` as additions to `_EVENT_VIEW_DEFS` (for the new event types) plus standalone view builders for the cross-event analytical views. All `JSON_VALUE` reads of `attributes.adk.node.path` should be wrapped in `NULLIF(..., '')` because ADK defaults `node_info.path` to the empty string for non-workflow events.

1. **New SQL views** (parallel to existing per-event views). The SDK's actual view-deployment surface is `ViewManager` in `src/bigquery_agent_analytics/views.py` plus the CLI — that is where these view definitions belong. **If** persisted analytical tables are later needed, add a dedicated materializer for these views, borrowing patterns from `materialize_window.py` (which is window/ontology-specific, not a general view materializer). The `examples/migration_v5/periodic_materialization/` folder is a deploy example and not the primary SDK pattern.

**General identity rule for every cross-event ADK 2.0 view:** the BigQuery `agent_events` table is a multi-tenant aggregate (multiple `app_name`s and `user_id`s can share one table), so any view that joins / groups events across rows must include the full telemetry identity — `JSON_VALUE(attributes, '$.adk.app_name')`, `user_id`, and `session_id` — in its `GROUP BY` / `JOIN ON` clauses. Each view bullet below states the full grouping; do not regress to partial keys.
- `workflow_invocations`: one row per `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id, node.path)` reconstructing the DAG via the producer-serialized **`attributes.adk.node.parent_path`** (not `parent_run_id` — run ids repeat across branches and would corrupt the join). **Source: `attributes.adk.node` on originating-Event enriched rows.** Explicit filter `WHERE NULLIF(JSON_VALUE(attributes, '$.adk.node.path'), '') IS NOT NULL` — `NULLIF` alone makes the read null-safe but doesn't keep empty/non-workflow rows out of the grouping (`NodeInfo.path` defaults to `""` for non-workflow events per `events/event.py:45`), and without this filter Event-originating non-workflow rows would produce a meaningless null-node DAG row. Pre-2.0 rows fall out of the same filter.
- `agent_transfer_chains`: ordered transfer graph per `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id)`, from `AGENT_TRANSFER` events.
- `branch_fanout`: **first pass = cardinality only** — per `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id)`, count distinct `attributes.adk.branch` values. **Gated to Event-originating rows via `WHERE JSON_VALUE(attributes, '$.adk.source_event_id') IS NOT NULL`** (per #214) — same pattern as `scope_cardinality`. Excludes pre-2.0 rows and non-Event-originating v2 callback rows. **Zero-branch policy:** v2 Event-originating invocations with no `attributes.adk.branch` on any row produce a row with `distinct_branch_count = 0`, not absent, so "% of invocations that branch" queries work. Concurrency is intentionally deferred — a meaningful concurrency metric requires interval data from `WORKFLOW_NODE_STARTING/COMPLETED`, which is itself still design-required. Add a `branch_concurrency` follow-up view gated on the node-boundary design landing.
- `long_running_tool_durations`: pair `TOOL_PAUSED` → `TOOL_COMPLETED` on the **full key** — `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, JSON_VALUE(attributes, '$.adk.function_call_id'))` — not on `function_call_id` alone. **Filter to `attributes.adk.pause_kind = 'tool'` on both rows** (the discriminator is now required on `TOOL_COMPLETED` too — see producer §6) so HITL synthetic waits never enter tool-duration analytics. **Healthy-pair branch additionally excludes orphans:** the completed stream must filter `COALESCE(SAFE_CAST(JSON_VALUE(attributes, '$.adk.pause_orphan') AS BOOL), FALSE) = FALSE` so a completion marked orphan at write time (because the paused row wasn't yet visible) can't *also* be joined into a healthy pair once the paused row becomes visible to a later query. Without this exclusion the same `TOOL_COMPLETED` could appear as both a healthy duration and an orphan. Emit `MIN(pause_ts) → MAX(complete_ts)` elapsed time. Do not rely on `invoke_node` span duration — paused tools can cross invocation boundaries, which would make the comparison incorrect. **Dedupe runs per stream** (separate paused-dedup / completed-dedup CTEs, or include `event_type` in the partition) — a single `ROW_NUMBER` partition across both event types would drop the matching completion row under a duplicate paused row. **Orphan branch:** `TOOL_COMPLETED` with `pause_orphan = true AND pause_kind = 'tool'` appears as rows with `status = 'orphan_completion'`, null `pause_ts`, and the completion timestamp in `complete_ts` — HITL orphans stay out via the same `pause_kind = 'tool'` gate. This keeps the producer's drift-quantification contract visible in the same view operators query for duration data; without it, orphans would silently fall out of a `PAUSED → COMPLETED` join.
- `compaction_windows`: per `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id)`, the ranges where rows were collapsed, derived from `EVENT_COMPACTION.content.{start_timestamp, end_timestamp}`. **Unit conversion is part of the contract:** ADK exposes those fields as float epoch seconds (`event_actions.py:42-45`; matching `Event.timestamp` at `event.py:156`), while BQAA's top-level `timestamp` column is a BigQuery `TIMESTAMP`. The view must expose typed `start_ts` and `end_ts` as `TIMESTAMP` values that preserve fractional seconds — i.e. `TIMESTAMP_MICROS(CAST( * 1000000 AS INT64))` rather than `TIMESTAMP_SECONDS()`, which would truncate sub-second windows. Optionally retain `start_seconds` / `end_seconds` (raw `FLOAT64`) for diagnostics. A fixture with fractional timestamps must demonstrate that a sub-second window doesn't collapse to a whole-second range. Dashboards can mark "data thinning" intervals.
- `scope_cardinality`: **one row per `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id, scope_kind)`**, where `scope_kind` is `COALESCE(JSON_VALUE(attributes, '$.adk.scope.kind'), 'unscoped')` so legitimately-unscoped rows appear under a synthetic `scope_kind = 'unscoped'`. **Value columns** measured in the right units — one ADK `Event` already fans out into multiple BQAA rows on `on_event_callback` (`STATE_DELTA`, HITL, A2A, `AGENT_RESPONSE` can co-emit from one source Event, see `bigquery_agent_analytics_plugin.py:2991`, `:3001`, `:3067`, `:3109`), so a naive `COUNT(*)` measures emitted analytics rows, not Events:
- `distinct_scope_count = COUNT(DISTINCT JSON_VALUE(attributes, '$.adk.scope.id'))` — cardinality of scope IDs (meaningful for scoped kinds; null/0 for `unscoped`).
- `source_event_count = COUNT(DISTINCT JSON_VALUE(attributes, '$.adk.source_event_id'))` — distinct ADK Events. This is the volume metric to chart against `scope_kind`.
- `row_count = COUNT(*)` — diagnostic only, retained so consumers can spot fan-out anomalies. Do not use as the primary volume metric.
- **Gate the view to Event-originating rows via `WHERE JSON_VALUE(attributes, '$.adk.source_event_id') IS NOT NULL`** — `schema_version IS NOT NULL` is too broad because non-Event-originating callback rows (`USER_MESSAGE_RECEIVED`, before-model, before-tool, etc.) are also ADK-enriched but have no meaningful `Event.isolation_scope` to bucket, so they would inflate the `unscoped` count. Pre-2.0 rows (no `attributes.adk` block) are excluded by the same gate, since they have no `source_event_id` either. `'unscoped'` here means "v2 Event-originating row that ADK marked as legitimately unscoped," nothing else.
2. **Optional `bqaa_adk.py` adapter inside this SDK** (parallel to whatever ADK-specific surface lands as the producer story stabilizes). Thin — just translates ADK plugin callbacks into logger calls. **Decision deferred**: only worth adding if the project decides the SDK should ship a parallel producer to the in-repo ADK plugin; otherwise it's redundant with the upstream plugin.
3. **Schema migration doc**: document the `attributes.adk.*` contract as the supported producer-consumer interface, version it (`attributes.adk.schema_version`), and define a deprecation policy.
4. **Query examples** in `USER_GUIDE.md` for the five most common ADK 2.0 questions (workflow tree, transfer chain, long-running tool wall-clock, branch fanout, compaction-aware token totals).

## Acceptance criteria

- After the `EventData` plumbing sub-issue lands, **rows with an originating ADK `Event`** carry `attributes.adk.node.path` (raw, possibly `""`), `attributes.adk.node.parent_path` (JSON `null` for root/bare nodes — single canonical no-parent form), `attributes.adk.branch`, and either `attributes.adk.scope = {id, kind}` for scoped rows or `attributes.adk.scope = null` for unscoped (the single canonical form). Rows without an originating Event leave these null — the consumer views handle that explicitly.
- `attributes.adk.schema_version` and `attributes.adk.app_name` are present on **every** ADK-enriched row.
- `attributes.adk.source_event_id` is present **only on rows with an originating ADK `Event`**.
- The representative workflow fixture covers all scope shapes: full-path node scope (`/@`), bare node scope (`name@`), an ADK-generated `adk-...` function-call-id scope, a **model-provided non-`adk-` function-call-id scope** (e.g. `call_abc123` or `toolu_xyz`), and unscoped — and consumer tests assert the correct `scope.kind` (or `scope = null`) for each. Bare node scopes must classify as `node_run` (v4 bug guard); model-provided FC IDs must classify as `function_call` without depending on prefix shape (v10 bug guard).
- **Six** new event types (`AGENT_TRANSFER`, `EVENT_COMPACTION`, `AGENT_STATE_CHECKPOINT`, `WORKFLOW_NODE_STARTING`, `WORKFLOW_NODE_COMPLETED`, **`TOOL_PAUSED`**) appear for a representative multi-agent run. `AGENT_STATE_CHECKPOINT` fires for at least one `{agent_state: null, end_of_agent: true}` event and at least one `{agent_state: {...}, end_of_agent: false}` event in the fixture. All six are recognized across the **full SDK event-type surface** — `EventType` in `trace.py`, `event_semantics.py` categorization, `trace_evaluator.py::_DEFAULT_EVENT_TYPES`, UDF label maps (e.g. `udf_kernels.py`), **and `_EVENT_VIEW_DEFS` (typed per-event-type views)**. Typed extra columns for `AGENT_TRANSFER`, `EVENT_COMPACTION`, `AGENT_STATE_CHECKPOINT` (inline), and `TOOL_PAUSED` are specified per #211 Section A; **typed extra columns for `WORKFLOW_NODE_STARTING` / `WORKFLOW_NODE_COMPLETED` are blocked on #207** and may ship as base-header-only views with a TODO link until that lands. The existing `TOOL_COMPLETED` typed-view shape decision from #211 Section B (extend with `attributes.adk.{function_call_id, pause_kind, pause_orphan}` columns, or document that consumers read those from `long_running_tool_durations` / base-table JSON) must be made and reflected in `SDK.md` (#220).
- `TOOL_PAUSED` **and** long-running-path `TOOL_COMPLETED` (both paired completions and orphans) carry `attributes.adk.pause_kind`. `TOOL_PAUSED` and `TOOL_COMPLETED` match on the **full telemetry key** — `(attributes.adk.app_name, user_id, session_id, attributes.adk.function_call_id)` — not on `function_call_id` alone, and `long_running_tool_durations` (filtered to `pause_kind = 'tool'` on both paired and orphan rows) reconstructs end-to-end wall-clock time from timestamp pairing on that full key — **including** runs where the pause spans multiple invocations. **Healthy-pair stream excludes orphans**: the completed stream in the healthy-pair branch requires `COALESCE(SAFE_CAST(JSON_VALUE(attributes, '$.adk.pause_orphan') AS BOOL), FALSE) = FALSE`, so a completion tagged orphan at write time (paused row not yet visible) cannot also appear as a healthy duration when the paused row becomes visible to a later query. **Orphan branch**: `TOOL_COMPLETED` with `pause_orphan = true AND pause_kind = 'tool'` surfaces as `status = 'orphan_completion'`, null `pause_ts` — orphans appear only here, not in healthy pairs. HITL synthetic pauses are excluded throughout. **HITL-orphan defensive test**: a `TOOL_COMPLETED` row with `pause_kind = 'hitl_confirmation'` and `pause_orphan = true` is **inserted synthetically into the consumer fixture (direct BigQuery insert, NOT producer-emitted — per #199 HITL completions stay on the `HITL_*_COMPLETED` flow)** and asserted to be absent from `long_running_tool_durations`. This proves the consumer view's filter is defensive against malformed/foreign rows.
- The six SQL views run on a real workflow trace and return expected row counts. **All six cross-event analytical views are deployed by one supported SDK deployment path** — `ViewManager.create_all_views()` (or the chosen sibling manager) plus the CLI equivalent — in a single invocation. Acceptance is not passed by view-definition code that no public command creates.
- `long_running_tool_durations` includes orphan-completion rows (`status = 'orphan_completion'`, null `pause_ts`) for every `TOOL_COMPLETED` where `attributes.adk.pause_orphan = true` **AND `attributes.adk.pause_kind = 'tool'`**. HITL waits/completions are excluded from this view; real HITL behavior is analyzed from the typed `HITL_*` event views (per #199 the producer never emits HITL completions as `TOOL_COMPLETED` — they stay on the `HITL_*_COMPLETED` flow). The synthetic HITL-tagged `TOOL_COMPLETED` fixture row from #218 is defensive malformed/foreign-row coverage only and is not a metric source. Drift in the tool-pause path is observable from the same view as healthy pauses.
- `scope_cardinality` is gated on `attributes.adk.source_event_id IS NOT NULL` — both pre-2.0 rows and non-Event-originating v2 rows (`USER_MESSAGE_RECEIVED`, before-model, before-tool, etc.) are excluded. The `'unscoped'` bucket counts only Event-originating v2 rows where ADK legitimately emitted `scope = null`. Three value columns in correct units: `distinct_scope_count` (cardinality), `source_event_count` (distinct Events — primary volume metric), `row_count` (BQAA-row diagnostic only). A test in which one ADK `Event` emits two BQAA rows must show `source_event_count = 1, row_count = 2`.
- Pre-2.0 rows (no `attributes.adk` block) continue to query without errors — every new view uses `JSON_VALUE` / `JSON_QUERY` with `NULLIF` / `COALESCE` guards.

## Back-compat & risk

- Schema stays at the existing 16 top-level columns; only `attributes` JSON grows. No BigQuery migration job needed.
- `attributes.adk.schema_version` lets consumers gate on producer version.
- Pre-2.0 in-repo plugin rows continue to be readable by the new views (fields just resolve to NULL after `NULLIF`).
- Risk: state snapshots in `AGENT_STATE_CHECKPOINT` can be large. The producer must cap via a size env var and overflow to GCS, but the **exact contract is deferred to the dedicated `AGENT_STATE_CHECKPOINT` offload design sub-issue** — `HybridContentParser` is content-part / text-shaped and not directly reusable for a structured-dict snapshot, so URI placement, SHA-256 location, inline removal/replacement, and shared-vs-new helper decisions all need to be made in that sub-issue before implementation.
- Risk: `WORKFLOW_NODE_STARTING/COMPLETED` is the highest-design-risk item — `BasePlugin` has no node lifecycle hook, so either we consume OTel spans (extra dependency, but accurate) or derive from event observation (simpler, but boundary detection is fuzzy on no-event nodes). Resolve in a dedicated sub-issue before implementation.

## Sub-issues to file from this tracker

Producer (against `adk-python` or vendored into this repo):

- [ ] **Prerequisite plumbing** (#194): thread either the source `Event` reference, **or** a typed telemetry context struct carrying at minimum `event_id`, `node_info`, `branch`, `isolation_scope`, `actions`, `long_running_tool_ids`, `content`, `timestamp`, `custom_metadata`, into `EventData` so `_log_event` can enrich rows that don't originate from `on_event_callback`. A literal `NodeInfo + EventActions` is too narrow — see #194 for the full mapping of struct fields to downstream sub-issues. Document the resulting coverage matrix per callback.
- [ ] Enrich `_log_event` with `attributes.adk.node` — serialize `path`, the path-derived `run_id`, and the producer-computed **`parent_path`** (canonical DAG join key). `parent_path` is everything before the final `/@` when `path` contains a `/`; otherwise (bare path or empty path) emit JSON `null`. `parent_run_id` may be included as a debugging-convenience field only. Best-effort until the plumbing lands.
- [ ] Enrich every originating-Event row with `attributes.adk.branch`.
- [ ] Define producer-owned `attributes.adk.scope` — `null` for unscoped (single canonical form), `{id, kind}` for scoped with `kind ∈ {function_call, node_run, unknown}` (no `root` kind — current ADK emits `None`, not a root marker). Implement shape-derivation in this order: (1) `None` → `null`; (2) `name@` or `/@` → `node_run`; (3) **any non-empty string that didn't match step 2** → `function_call` (default — do not depend on prefix shape; model-provided FC IDs like `call_*` and `toolu_*` are legitimate and must classify as `function_call`); (4) empty/non-string anomalies → `unknown` with a producer warning log. Document that raw `isolation_scope` is internal/unstable and not part of the public contract.
- [ ] Stamp `attributes.adk.schema_version` on every ADK-enriched row.
- [ ] Stamp `attributes.adk.app_name` (from `InvocationContext.app_name`) on every ADK-enriched row. Required for the pause registry key and the general cross-event view identity rule. Do not promote to a top-level column.
- [ ] Stamp `attributes.adk.source_event_id` on every row that originates from an `Event` (reliable join key against ADK OTel `associated_event_ids`). Leave null on rows that do not originate from an `Event`.
- [ ] Emit `AGENT_TRANSFER` from `actions.transfer_to_agent` (`{from_agent, to_agent}`; reason optional).
- [ ] Emit `EVENT_COMPACTION` from `actions.compaction` (`{start_timestamp, end_timestamp, compacted_content}`).
- [ ] Emit `AGENT_STATE_CHECKPOINT` whenever `actions.agent_state is not None` **or** `actions.end_of_agent is True` (either field independently). Allow `{agent_state: null, end_of_agent: true}` payloads.
- [ ] **Design**: define the oversized-state offload contract for `AGENT_STATE_CHECKPOINT`. `HybridContentParser` is content-part / text-shaped and is not directly reusable for a structured dict. Decide: (a) where the GCS URI lives (`attributes.adk.agent_state_uri`?), (b) where the SHA-256 lives (`attributes.adk.agent_state_sha256`), (c) whether the inline `content.agent_state` is removed or replaced with a stub, (d) whether to share parser code or add a dedicated helper. Document the size cap env var.
- [ ] **Design**: choose derivation strategy for `WORKFLOW_NODE_STARTING/COMPLETED` — OTel span consumption vs. event-observation. Implement after the decision lands.
- [ ] Emit `TOOL_PAUSED` for each `long_running_tool_id`. Required: `attributes.adk.pause_kind ∈ {tool, hitl_credential, hitl_confirmation, hitl_input}`. **Derivation (id→name, per #199):** for each `long_running_tool_id`, locate the matching `part.function_call.id` in the source event's `content.parts`, then look up `_HITL_EVENT_MAP[part.function_call.name]` (line 87) → matching `hitl_*` kind or `'tool'`. **Do not look up `_HITL_EVENT_MAP` against the id value itself** — that would classify every HITL pause as `'tool'`. If no matching `part.function_call` is found, emit `pause_kind = 'tool'` with a warning log. **`pause_kind` is also required on long-running-path `TOOL_COMPLETED` rows** (both healthy completions and orphans) — derive on the same iteration that emits the completion (`bigquery_agent_analytics_plugin.py:3022`) using the same `_HITL_EVENT_MAP[function_response.name]` lookup. Without this, orphan completions have no paired `TOOL_PAUSED` to inherit the kind from and HITL orphans would contaminate `long_running_tool_durations`. Propagate `attributes.adk.function_call_id` onto both `TOOL_PAUSED` and `TOOL_COMPLETED` rows.
- [ ] Implement the pause registry per the contract above: composite key `(app_name, user_id, session_id, function_call_id)`; **durable source of truth is BigQuery `TOOL_PAUSED` rows + ADK in-session history** (any process-local cache is a latency-optimization layer in front of those sources, not a substitute); `BQAA_PAUSE_REGISTRY_TTL_HOURS` TTL; at-most-once `TOOL_COMPLETED` emission. **`pause_orphan = true` only after the chosen visibility strategy (cache+BQ fallback, in-session reconstruction, or settling-time grace window) fails to find a match** — never on a bare immediate-BQ-miss. **Detect non-HITL `function_response.id` completions in `on_event_callback` / `on_user_message_callback`** (not only `after_tool_callback`), reusing the iteration already present at `bigquery_agent_analytics_plugin.py:3022`.
- [ ] **Design**: resolve the BigQuery streaming-visibility caveat for the pause registry (eventual-visibility can produce false `pause_orphan` flags on immediate read-after-streaming-insert). Pick one of: in-process cache + BigQuery durable fallback, in-session ADK history reconstruction, or a settling-time grace window. Include a test that streams a `TOOL_PAUSED` and then immediately observes a matching `function_response` without falsely orphaning.
- [ ] Producer or explicit deferral: capture `actions.route`, `actions.render_ui_widgets`, and `actions.rewind_before_invocation_id`.
- [ ] Delete the deprecated `on_state_change_callback` stub.
- [ ] **Design**: OTel correlation strategy. Decide whether `attributes.adk.otel_span_id` is reliable enough to ship (current proposal: best-effort) or whether the consumer should join via `attributes.adk.source_event_id` ↔ ADK's span-side `associated_event_ids` instead. Defer the eventual OTel exporter to a separate tracking item.

Consumer (this repo, mostly under `src/bigquery_agent_analytics/`):

- [ ] **Audit every SDK event-type allowlist / categorization surface and register the six new types** (`AGENT_TRANSFER`, `EVENT_COMPACTION`, `AGENT_STATE_CHECKPOINT`, `WORKFLOW_NODE_STARTING`, `WORKFLOW_NODE_COMPLETED`, `TOOL_PAUSED`). Concrete surfaces (audit the SDK and extend if more are found): `src/bigquery_agent_analytics/views.py::_EVENT_VIEW_DEFS` **(typed per-event-type views — register all six, but per #211 the typed-column shapes for `WORKFLOW_NODE_STARTING/COMPLETED` are blocked on #207's node-boundary derivation decision)**, `src/bigquery_agent_analytics/trace.py::EventType`, `event_semantics.py` categorization, `trace_evaluator.py::_DEFAULT_EVENT_TYPES`, and any UDF label maps (e.g. `udf_kernels.py`). Also covered by #211: decide whether the **existing `TOOL_COMPLETED` typed view** exposes the new long-running ADK fields (`function_call_id`, `pause_kind`, `pause_orphan`) or whether `SDK.md` (#220) documents that consumers read those from `long_running_tool_durations` / base-table JSON.
- [ ] **Cross-event view deployment plumbing** — the existing `ViewManager` (`src/bigquery_agent_analytics/views.py:281`) and CLI `views` commands (`cli.py:1659`) are per-event-type only, keyed off `_EVENT_VIEW_DEFS`. The six new analytical views (`workflow_invocations`, `agent_transfer_chains`, `branch_fanout`, `long_running_tool_durations`, `compaction_windows`, `scope_cardinality`) are cross-event and don't fit that shape. Decide: extend `ViewManager` with a cross-event view registry, or add a sibling `AnalyticsViewManager`. Acceptance: `create_all_views()` (and the CLI equivalent) must deploy all six cross-event views in addition to the per-event-type views, in one invocation.
- [ ] View: `workflow_invocations` — grouped on the **full telemetry identity** `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id, node.path)`; DAG join on **`attributes.adk.node.parent_path`** (not `parent_run_id`); explicit `WHERE NULLIF(JSON_VALUE(attributes, '$.adk.node.path'), '') IS NOT NULL` filter so non-workflow / pre-2.0 rows don't produce a null-node DAG row. Null-safe test asserts non-workflow rows (default `path = ""`) are excluded from the DAG.
- [ ] View: `agent_transfer_chains` — grouped on the **full telemetry identity** `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id)`.
- [ ] View: `branch_fanout` — **first pass = cardinality only**. Per `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id)`, count distinct `attributes.adk.branch`. **Gated to Event-originating rows via `WHERE JSON_VALUE(attributes, '$.adk.source_event_id') IS NOT NULL`** (same pattern as `scope_cardinality`). **Zero-branch row policy**: v2 Event-originating invocations with no branch produce a row with `distinct_branch_count = 0`, not absent. Defer `branch_concurrency` as a follow-up view gated on the workflow-node-boundary design decision (#207).
- [ ] View: `long_running_tool_durations` — timestamp pairing on the **full key** `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, JSON_VALUE(attributes, '$.adk.function_call_id'))`; filtered to `pause_kind = 'tool'` on both paired rows; **do not** use `invoke_node` span duration. **Healthy-pair completed stream additionally excludes `pause_orphan = true`** via `COALESCE(SAFE_CAST(JSON_VALUE(attributes, '$.adk.pause_orphan') AS BOOL), FALSE) = FALSE` (per #215 — without this, a completion tagged orphan at write time can be joined to a now-visible `TOOL_PAUSED` and appear as a healthy duration AND an orphan). **Dedupe runs per stream** (separate paused / completed CTEs, or include `event_type` in the partition — a single-partition `ROW_NUMBER` across both event types would drop the matching completion row under a duplicate paused row). **Orphan branch**: `TOOL_COMPLETED` with `pause_orphan = true AND pause_kind = 'tool'` included with `status = 'orphan_completion'`, null `pause_ts`, completion timestamp in `complete_ts`.
- [ ] View: `compaction_windows` — grouped on the **full telemetry identity** `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id)`. **Convert ADK float epoch seconds to BQ `TIMESTAMP` via `TIMESTAMP_MICROS(CAST( * 1000000 AS INT64))`** (not `TIMESTAMP_SECONDS`, which truncates fractional windows). Expose `start_ts` / `end_ts` as `TIMESTAMP`; optional `start_seconds` / `end_seconds` as `FLOAT64` for diagnostics. Fixture with fractional timestamps must show sub-second windows survive the conversion.
- [ ] View: `scope_cardinality` — one row per `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id, scope_kind)` where `scope_kind = COALESCE(JSON_VALUE(attributes, '$.adk.scope.kind'), 'unscoped')`. **Three value columns** measured in the right units: `distinct_scope_count = COUNT(DISTINCT scope.id)` (cardinality), `source_event_count = COUNT(DISTINCT source_event_id)` (volume — distinct Events, not BQAA rows), `row_count = COUNT(*)` (diagnostic-only — one Event fans out to multiple BQAA rows in `on_event_callback`, so `COUNT(*)` lies as a volume metric). **Gate to Event-originating rows** via `WHERE JSON_VALUE(attributes, '$.adk.source_event_id') IS NOT NULL` — not `schema_version IS NOT NULL`. Null-safe test asserts pre-2.0 rows and non-Event-originating v2 rows are both excluded. **Fan-out test**: one source `Event` that emits two BQAA rows (e.g. `STATE_DELTA` + `AGENT_RESPONSE` from the same Event) must produce `source_event_count = 1, row_count = 2`.
- [ ] Null-safe tests for pre-2.0 rows (no `attributes.adk` block) against every new view.
- [ ] Representative workflow fixture covering branch fanout, transfer, compaction, checkpoint (both `{agent_state: null, end_of_agent: true}` and `{agent_state: {...}, end_of_agent: false}` shapes), a long-running pause/resume that spans invocations, **and all scope shapes** — full-path node scope (`/@`), bare node scope (`name@`), ADK-generated `adk-...` function-call-id scope, **model-provided non-`adk-` function-call-id scope** (e.g. `call_abc123`, `toolu_xyz`), and unscoped (`scope = null`).
- [ ] Document `attributes.adk.*` contract + `schema_version` in `SDK.md` or `docs/`.
- [ ] Decide on optional `bqaa_adk.py` SDK adapter (yes/no based on ADK-repo direction).
- [ ] `USER_GUIDE.md` ADK 2.0 query recipes.

## References

- ADK 2.0 event model: `src/google/adk/events/event.py`, `event_actions.py`.
- In-repo plugin: `src/google/adk/plugins/bigquery_agent_analytics_plugin.py` (callbacks at 2910–3507; `on_event_callback` at 2956; `_log_event` at 2815; deprecated `on_state_change_callback` at 3125; schema columns at 1791–1804).
- Standalone SDK on `main`: `src/bigquery_agent_analytics/views.py` (`_EVENT_VIEW_DEFS`, `ViewManager`); related modules under `src/bigquery_agent_analytics/` include `trace.py`, `materialize_window.py`, `evaluators.py`.
- ADK 2.0 OTel: `src/google/adk/telemetry/` (semconv v1.36/v1.41).