adk-2.0 producer: minimum plan to unblock mid-June customer cutover

> **Revision history**
> - v5: References-section cleanup — added #200/#201/#202 (the existing emit sub-issues for C4/C5/C6, parallel to how #196/#197/#198 are cited for C1/C2/C3) and inline `→ maps to **#NNN**` pointers on the C4/C5/C6 bullets. The #199 reference was rewritten to make explicit that #293 covers **only the pair-key subset of #199**; the pause registry and `pause_orphan` semantics defer to #206. No contract changes.
> - v4: wording-only corrections from the v3 review — (1) C7 prose no longer claims the future registry will *retroactively* mark mid-June orphans; replaced with an honest "future rows only; historical rows remain pair-key-only unless a separate backfill is designed" framing. (2) v3 revision-history bullet rewritten to stop calling #206 a C7 blocker — the dependency graph already correctly says `NOT blocked by #206`; the bullet now matches that, framing #206 as the deferred home for orphan semantics rather than a blocker. No contract changes.
> - v3: applied review corrections — (1) **C7 narrowed for the mid-June bar**: emit `TOOL_PAUSED` and non-HITL `TOOL_COMPLETED` with the pairable keys (`function_call_id`, `pause_kind`); the pause registry's read-after-write visibility semantics (`pause_orphan`, settling delay vs in-process cache vs in-session reconstruction) **defer to #206**, which is now named as the deferred home for orphan semantics (C7 itself is **not** blocked by #206 — see the dependency graph). The customer's mid-June cutover needs row-pair join keys, not orphan semantics. (2) **C8 attribute path realigned to flat-with-prefix** to match #203 and the entire envelope convention in #190 (`attributes.adk.branch`, `attributes.adk.scope`, `attributes.adk.app_name`, `attributes.adk.source_event_id`, `attributes.adk.node`, `attributes.adk.pause_kind` — all flat). C8 now reads `attributes.adk.route`, `attributes.adk.render_ui_widgets`, `attributes.adk.rewind_before_invocation_id` (was: nested under `attributes.adk.actions.*`). Acceptance #4 and the test plan updated to match. (3) Reviewer's HIGH on `AGENT_STATE_CHECKPOINT` blank `event.id` was **rejected with verification**: `Event.model_post_init` (`events/event.py:271-275`) auto-assigns `id = Event.new_id()` for any Event constructed without one, including the `_create_agent_state_event` path (`agents/base_agent.py:213-218`). Empirically verified: `Event(invocation_id='inv-1', author='agentA', branch='b', actions=EventActions()).id` is a 36-char UUID immediately after construction, before the runner hands the Event to `on_event_callback`. No stabilization rule needed; C6 rows have real `source_event_id`s.
> - v2: applied review corrections — (1) **C7 HITL routing fix**: long-running-path `TOOL_COMPLETED` is non-HITL-only and always carries `pause_kind = 'tool'`; HITL function responses stay on the existing `HITL_*_COMPLETED` stream (verified against `bigquery_agent_analytics_plugin.py:2944,3029`), not `TOOL_COMPLETED`. `TOOL_PAUSED` may still carry HITL `pause_kind`s. (2) **Envelope contract split**: A1/A2 (`schema_version`, `app_name`) stamp on every ADK-enriched row regardless of origin; A3/C1/C2/C3 (`source_event_id`, `node`, `branch`, `scope`) only on rows with a real originating Event or typed telemetry context — no fabricated identity for callback-only rows. (3) **Producer null-safety reframed**: tests now cover default-empty `NodeInfo.path = ""` (`events/event.py:45`), absent `branch`/`isolation_scope` staying JSON null, and explicit no-fabricated-node behavior — historical pre-2.0 table rows are the consumer's problem (#190/#211), not this issue's. (4) **`AGENT_TRANSFER.from_agent` source pinned**: `from_agent = event.author`, `to_agent = event.actions.transfer_to_agent` (verified `event_actions.py:75` carries the target only). (5) Typo: deferred OTel row now correctly references A3 (`source_event_id`), not C3. (6) Precision edits: C5 compaction timestamps must preserve fractional float-epoch-second precision in the producer (view conversion deferred); C1 gets an explicit `NodeInfo.path == ""` fixture; acceptance #3 SQL key spelled with JSON paths (`JSON_VALUE(attributes, '$.adk.app_name')`, `JSON_VALUE(attributes, '$.adk.function_call_id')`).

> Spawned from the customer-needs comment on #190: https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/190#issuecomment-4636455013
> Scope: **producer side only** (the BQ AA Plugin in `google/adk-python`, `src/google/adk/plugins/bigquery_agent_analytics_plugin.py`). Consumer SDK views in #190 are not on this critical path — once the producer writes the rows, the customer can query them directly via SQL while the typed views land separately.

## Why this issue exists

#190 is the long-form ADK 2.0 tracking issue (28 sub-issues, producer + consumer). The customer pinned a comment listing the specific fields they need before they take ADK 2.0 to production mid-June:

- `event.long_running_tool_ids`
- `event.node_info`
- `actions.compaction`
- `actions.transfer_to_agent`
- `actions.end_of_agent` / `agent_state`
- `actions.route` / UI / `rewind`
- Workflow boundaries

This issue is a **minimum**, scoped subset of #190 that maps 1:1 to that list, ordered so the customer can ship without waiting for the full plan (workflow-node-boundary derivation, OTel correlation, optional adapters, consumer views, pause-orphan registry semantics — all deferred).

## Producer state today

Already emitted (no work): `INVOCATION_STARTING/COMPLETED`, `AGENT_STARTING/RESPONSE/COMPLETED`, `USER_MESSAGE_RECEIVED`, `LLM_REQUEST/RESPONSE/ERROR`, `TOOL_STARTING/COMPLETED/ERROR`, `STATE_DELTA`, `HITL_CREDENTIAL_REQUEST/HITL_CREDENTIAL_REQUEST_COMPLETED`, `HITL_CONFIRMATION_REQUEST/HITL_CONFIRMATION_REQUEST_COMPLETED`, `HITL_INPUT_REQUEST/HITL_INPUT_REQUEST_COMPLETED`.

Missing for the customer ask: every item below.

## Minimum plan

### Group A — envelope every enriched row carries (blocks everything else)

A1 and A2 stamp on **every ADK-enriched row** regardless of whether the row originates from an `Event`. A3 only stamps on rows with a real originating `Event` (or a future typed telemetry context that carries an equivalent identity). The producer must not synthesize a `source_event_id` for callback-only rows; consumers can detect Event-originating rows via `WHERE JSON_VALUE(attributes, '$.adk.source_event_id') IS NOT NULL`. **Note**: any Event constructed without an `id` (e.g. `_create_agent_state_event` in `agents/base_agent.py:213-218`) is auto-stabilized by `Event.model_post_init` (`events/event.py:271-275`), which sets `self.id = Event.new_id()` before the plugin sees it — no additional producer-side stabilization needed.

- [ ] **A1. `attributes.adk.schema_version`** — constant on every ADK-enriched row. Single source of truth for "this row was produced by an ADK 2.0-aware plugin build".
- [ ] **A2. `attributes.adk.app_name`** — from `InvocationContext.app_name`, on every ADK-enriched row. Required for the pause registry composite key and for the general cross-event identity rule.
- [ ] **A3. `attributes.adk.source_event_id`** — only on rows that originate from an `Event` (or a typed telemetry context carrying an equivalent identity, per B0). **Leave null** on rows produced by callbacks that don't receive an Event from the framework. Reliable join key against ADK OTel `associated_event_ids`. **Never fabricate.**

### Group B — prerequisite plumbing (blocks A3 + C1/C2/C3 for non-`on_event_callback` paths)

- [ ] **B0. Thread the source `Event` (or a typed telemetry context) into `EventData`** so `_log_event` can enrich rows that don't originate from `on_event_callback`. Today's `EventData` (`bigquery_agent_analytics_plugin.py:1956`) carries span/latency/model fields only and no source `Event`. Covered in detail by **#194** — this issue depends on #194 landing first or in parallel. **Deliverable: a per-callback coverage matrix** that explicitly enumerates, for every callback the plugin implements, whether A3/C1/C2/C3 enrichment is feasible (real Event reachable), best-effort, or null (no Event from framework). This issue's acceptance criteria only require enrichment where the matrix says it's feasible.

### Group C — customer-prioritized event/action capture

Each maps to one bullet in the customer's comment. Acceptance for each = a fixture turn that exercises the path produces the expected BigQuery row(s) **on rows the B0 matrix marks as Event-originating**.

- [ ] **C1. `attributes.adk.node = {path, run_id, parent_path}`** on Event-originating rows. `parent_path` is everything before the final `/@` when `path` contains a `/`; otherwise JSON `null`. **`NodeInfo.path` defaults to `""` (`events/event.py:45`)** — the producer must emit the empty string verbatim (or a documented null form) and **must not synthesize a fake workflow node or parent path** from it. → maps to **#196**. Customer ask: `event.node_info`.
- [ ] **C2. `attributes.adk.branch`** on Event-originating rows. **Absent branch stays JSON null**, never a sentinel like `""` or `"default"`. → maps to **#197**. Customer ask: implicit in the node-info neighborhood.
- [ ] **C3. `attributes.adk.scope = null | {id, kind}`** with the **v15 derivation rule** (#190 v5 fixed the misclassification bug): (1) `None` → `null`; (2) `name@` / `/@` → `node_run`; (3) any other non-empty string → `function_call`; (4) empty/non-string → `unknown` with a warning. **Absent `isolation_scope` stays `null`**, never a sentinel. → maps to **#198**. Customer ask: implicit — lets the customer separate node-loop fan-out from function-call fan-out.
- [ ] **C4. `AGENT_TRANSFER`** from `actions.transfer_to_agent`. **`EventActions.transfer_to_agent` stores the target agent only** (`event_actions.py:75`), so payload derivation is pinned: `to_agent = event.actions.transfer_to_agent`; `from_agent = event.author` on Event-originating transfer rows. If a non-Event path ever needs to emit a transfer row (none today), it must supply an equivalent typed source or leave `from_agent` null with a warning log — no fabricated authors. → maps to **#200**. Customer ask: `actions.transfer_to_agent`.
- [ ] **C5. `EVENT_COMPACTION`** from `actions.compaction` (`{start_timestamp, end_timestamp, compacted_content}`). **`start_timestamp` and `end_timestamp` are ADK float epoch seconds** (`events/event_actions.py` `EventCompaction` fields); the producer must preserve fractional precision in the emitted JSON (consumer view conversion is deferred per #190). → maps to **#201**. Customer ask: `actions.compaction`.
- [ ] **C6. `AGENT_STATE_CHECKPOINT`** when **either** `actions.agent_state is not None` **or** `actions.end_of_agent is True`. Allow `{agent_state: null, end_of_agent: true}` payloads. **Minimum: inline payload only.** GCS offload for oversized state is a separate design item (#190's open design checkbox) — file as a follow-up; don't gate the cutover on it. → maps to **#202**. Customer ask: `actions.end_of_agent` / `agent_state`.
- [ ] **C7. `TOOL_PAUSED` (+ paired long-running `TOOL_COMPLETED` enrichment)** from `event.long_running_tool_ids`. **Mid-June minimum bar: emit pairable rows only.** HITL routing is unchanged — HITL function responses continue routing to `HITL_*_COMPLETED` (`bigquery_agent_analytics_plugin.py:2944,3029`), not `TOOL_COMPLETED`. Therefore:
- `TOOL_PAUSED` may carry **any** `pause_kind ∈ {tool, hitl_credential, hitl_confirmation, hitl_input}` derived per #199's id→name lookup (`_HITL_EVENT_MAP[part.function_call.name]`, not against the id value).
- long-running-path `TOOL_COMPLETED` is **non-HITL only** and always carries `attributes.adk.pause_kind = 'tool'`. HITL completions remain on `HITL_CREDENTIAL_REQUEST_COMPLETED` / `HITL_CONFIRMATION_REQUEST_COMPLETED` / `HITL_INPUT_REQUEST_COMPLETED`.
- Required attributes on both `TOOL_PAUSED` and (non-HITL) long-running `TOOL_COMPLETED`: `attributes.adk.pause_kind`, `attributes.adk.function_call_id`. These are the pair keys the customer joins on in SQL.
- **`pause_orphan` and the registry's read-after-write visibility semantics (in-process cache vs settling-time delay vs in-session reconstruction) are out of scope for this issue — they defer to #206**, the dedicated design blocker. Customer SQL during the mid-June window will join `TOOL_PAUSED` ↔ `TOOL_COMPLETED` directly and accept the small set of unpaired rows that the eventual-visibility window can produce. The registry strategy lands later for future rows; historical rows emitted by this mid-June cut remain pair-key-only unless a separate backfill is explicitly designed.
Customer ask: `event.long_running_tool_ids`.
- [ ] **C8. Action attributes for `actions.route`, `actions.render_ui_widgets`, `actions.rewind_before_invocation_id`** — stamp under `attributes.adk.route`, `attributes.adk.render_ui_widgets`, `attributes.adk.rewind_before_invocation_id` (**flat-with-prefix**, matching #203's contract and the rest of the `attributes.adk.*` envelope in #190 — `branch`, `scope`, `node`, `app_name`, `source_event_id`, `pause_kind`, etc. are all flat) on the originating-Event row. **No new event types for the mid-June bar** — emitting them as attributes lets the customer filter/group without forcing a downstream consumer update. → covered by **#203**. Customer ask: `actions.route` / UI / `rewind`.

### Group D — cleanup

- [ ] **D1. Delete `on_state_change_callback`** (deprecated stub at `bigquery_agent_analytics_plugin.py:3131`, never called by ADK 2.0). Tiny but removes a misleading surface area for the customer's team when they read the plugin code.

## Explicitly deferred (post-cutover)

Each carries a one-line reason so the customer's team understands why their ask is not blocked:

| Deferred | Reason |
|---|---|
| `WORKFLOW_NODE_STARTING/COMPLETED` event types | Design-blocked in #190 / #207 (OTel-span vs event-observation). Workflow boundaries are **partially observable today** via `attributes.adk.node` (C1) on Event-originating rows, which is enough for the customer's first production query needs. The dedicated boundary events come after #207's decision lands. |
| OTel `attributes.adk.otel_span_id` (#205) | Best-effort only; consumer can join via **A3** `source_event_id` ↔ ADK's span-side `associated_event_ids` in the meantime. |
| Oversized-state GCS offload for `AGENT_STATE_CHECKPOINT` | Inline payloads cover all but very large state. File as follow-up if the customer's actual checkpoint sizes hit the limit. |
| Pause registry `pause_orphan` semantics + read-after-write visibility (#206) | Blocks the orphan-flag contract, not the row-pair join keys. The customer can compute long-running tool durations from direct `TOOL_PAUSED` ↔ `TOOL_COMPLETED` SQL joins during the mid-June window; orphan correctness lands when #206's strategy is chosen and the registry is added. |
| Optional `bqaa_adk.py` SDK adapter | Not customer-blocking. |
| Consumer SDK view registration (`#211`, `views.py`) for the new event types | Not customer-blocking — they can read base-table JSON until the typed views land. Tracked in #190. |
| Historical pre-2.0 table-row null-safety | Concerns rows already in the table that were written by an older producer. Belongs to consumer view work (#190 / #211), not this producer-only issue. |

## Dependency graph

```
B0 (#194) ───┬──► A3, C1, C2, C3, C4–C7
│ (only on B0-feasible callbacks)
A1, A2 ─┴──► every ADK-enriched row (no Event dep)

C4 AGENT_TRANSFER ◄── actions.transfer_to_agent
(from_agent=event.author)
C5 EVENT_COMPACTION ◄── actions.compaction
(preserve float precision)
C6 AGENT_STATE_CHECKPOINT ◄── actions.agent_state | end_of_agent
C7 TOOL_PAUSED + non-HITL TOOL_COMPLETED enrich
◄── event.long_running_tool_ids
NOT blocked by #206 (orphan semantics deferred)
C8 attributes.adk.{route, render_ui_widgets, rewind_before_invocation_id}
◄── actions.route / widgets / rewind
(covered by #203, same flat-with-prefix shape)

D1 — cleanup, no deps
```

## Acceptance criteria

A representative ADK 2.0 invocation that includes: agent transfer + event compaction + agent-state checkpoint (both `{agent_state: null, end_of_agent: true}` and `{agent_state: {...}, end_of_agent: false}` shapes) + a long-running pause that spans invocations + a `actions.route` or `rewind_before_invocation_id` action → produces BigQuery rows where, from the customer's SQL only:

1. **Every** ADK-enriched row exposes `attributes.adk.schema_version` and `attributes.adk.app_name`. **Event-originating rows additionally expose** `attributes.adk.source_event_id`, `attributes.adk.node`, `attributes.adk.branch`, `attributes.adk.scope` per the B0 coverage matrix; callback-only rows leave these JSON null without synthesis.
2. `AGENT_TRANSFER`, `EVENT_COMPACTION`, `AGENT_STATE_CHECKPOINT` rows are joinable by `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, invocation_id)`.
3. The customer can pair `TOOL_PAUSED` ↔ long-running `TOOL_COMPLETED` on `(JSON_VALUE(attributes, '$.adk.app_name'), user_id, session_id, JSON_VALUE(attributes, '$.adk.function_call_id'))` filtered to `JSON_VALUE(attributes, '$.adk.pause_kind') = 'tool'` on both rows, and compute long-running tool duration **without** the SDK's typed view and without the pause registry. Orphan rows are tolerated for the mid-June window; orphan correctness lands with #206. HITL completion durations are computed separately from the existing `HITL_*_REQUEST` → `HITL_*_REQUEST_COMPLETED` stream.
4. `actions.route` / `render_ui_widgets` / `rewind_before_invocation_id` are surfaced as `attributes.adk.route` / `attributes.adk.render_ui_widgets` / `attributes.adk.rewind_before_invocation_id` (flat-with-prefix, matching #203) on the originating Event row.
5. Producer-side tests stay green; new fixtures cover each emit path.
6. `on_state_change_callback` is gone from the public surface.

## Test plan (producer-only)

- Per-emit-path fixture for C4–C7 + C8.
- Envelope smoke (A1, A2) asserted on every row produced by an existing fixture.
- Envelope smoke (A3) asserted only on rows the B0 coverage matrix marks Event-originating; non-Event-originating callback rows asserted to leave A3 / C1 / C2 / C3 **JSON null** (not sentinel-stringed).
- **Empty-path fixture**: an Event with `NodeInfo.path == ""` must emit `attributes.adk.node.path = ""` (or a documented null form) and **must not** synthesize a workflow node or `parent_path`.
- **AGENT_STATE_CHECKPOINT id-stabilization smoke**: an `_create_agent_state_event`-shaped Event (constructed without `id=`) must arrive at the plugin with a non-empty `id`, and the resulting BigQuery row must carry that id in `attributes.adk.source_event_id`. (This is a regression guard for the Pydantic `model_post_init` auto-assign path; it does not require new producer logic.)
- `node` / `branch` / `scope` enrichment fixtures matching the shape-coverage list in #190 v5 (node-run scope, function-call scope, unscoped, model-provided FC IDs).
- C7 HITL non-routing assertion: a long-running pause whose function_call name is `adk_request_confirmation` (etc.) produces `TOOL_PAUSED` with `pause_kind = 'hitl_confirmation'`, and the matching function response produces a `HITL_CONFIRMATION_REQUEST_COMPLETED` row — **not** a `TOOL_COMPLETED` row.
- C7 pair-key assertion (no orphan logic): a non-HITL long-running pause + completion produces a `TOOL_PAUSED` and a `TOOL_COMPLETED` row with matching `function_call_id` and `pause_kind = 'tool'` on both. **No `pause_orphan` assertion** — that field's correctness defers to #206.
- C5 fractional-timestamp fixture: an `actions.compaction` with sub-second `start_timestamp` / `end_timestamp` must round-trip through the producer's JSON serialization with fractional precision intact.
- No new consumer SDK tests required — that work is tracked under #190 / #211 et al.

## Effort + ordering recommendation

Roughly in order of "what unlocks the next thing":

1. **A1, A2** — single-day-each; no deps; lands on every row immediately.
2. **B0 (#194)** — biggest single change; gates A3 and most of Group C.
3. **A3, C1, C2, C3** — fast once B0 lands; lots of overlap (same Event-enrichment path).
4. **C4, C5, C6 (inline), C8** — independent emit paths; can parallelize.
5. **C7** — mid-sized once narrowed to pair-key emit only (no registry); ships independently of #206.
6. **D1** — opportunistic cleanup, can ship in any PR.

If staffed seriously, A1–A3 + C1–C5 + C7 (pair-key) should be reachable by mid-June. C6 (inline) and C8 are small additions to the same PRs. Workflow node boundaries and pause-orphan semantics explicitly stay deferred.

## References

- #190 — full ADK 2.0 tracking issue (this is a scoped subset)
- #194 — prerequisite plumbing (EventData → source Event / telemetry context)
- #196, #197, #198 — Event-envelope sub-issues (C1/C2/C3) that this minimum plan covers
- #199 — long-running `TOOL_PAUSED` / `TOOL_COMPLETED` emit (C7). **#293 covers only the pair-key subset of #199** — `TOOL_PAUSED` with `pause_kind`, non-HITL long-running `TOOL_COMPLETED` with matching `function_call_id` + `pause_kind = 'tool'`. The pause registry, `pause_orphan` flag, and read-after-write visibility semantics from #199 are **deferred to #206** for the mid-June window.
- #200, #201, #202 — `AGENT_TRANSFER` (C4), `EVENT_COMPACTION` (C5), `AGENT_STATE_CHECKPOINT` (C6) emit sub-issues that this minimum plan covers
- #203 — `actions.route` / `render_ui_widgets` / `rewind_before_invocation_id` capture (C8's home)
- #206 — pause registry read-after-write design (C7's deferred orphan semantics)
- Customer comment: https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/190#issuecomment-4636455013
- Review v1 → v2: https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/293#issuecomment-4637233113
- Review v2 → v3: https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/293#issuecomment-4646344422
- Review v3 → v4: https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/293#issuecomment-4646396758
- Review v4 → v5: this thread