GoogleCloudPlatform / GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK

adk-2.0 post-cutover follow-ups: workflow boundaries, pause registry, offload, OTel, typed views

Open
#297 13 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
47
Forks
21
Avg merge
2d 13h
Merged PRs (30d)
33

Description

> **Status: v4 contract-stable as of 2026-06-08.** Selected approaches propagated into #206 / #207 / #208 / #211. #205 closed as superseded.
>
> This issue tracks the remaining work after #293 (mid-June minimum producer cut) lands. Five items; design decisions converged; assignees ready to pick up sub-issue bodies.

## Quick map (post-convergence)

| Item | Selected approach | Tracked at | Priority |
|---|---|---|---|
| Workflow boundary events | Event-observation derivation, invocation-segment scoped, pause-aware drain with ancestor propagation, no `WORKFLOW_NODE_PAUSED` | #207 (v4 body) | P1 |
| Pause registry / `pause_orphan` | Hybrid — `session_service.get_session(num_recent_events=None)` primary, BQ settling-time fallback on soft miss; never declares orphan on bare invocation-context miss | #206 (v4 body) | P0 |
| Oversized checkpoint GCS offload | Dedicated `_offload_structured_state` helper; `BQAA_AGENT_STATE_MAX_BYTES` default 512 KB; `_offloaded` stub; URI only at `attributes.adk.agent_state_uri`; sha256 at `attributes.adk.agent_state_sha256` | #208 (v3 body) | P2 |
| OTel correlation / span id | **Closed as superseded.** Reliable join is `attributes.adk.source_event_id` (#293) ↔ ADK span-side `gcp.vertex.agent.associated_event_ids` (`telemetry/node_tracing.py:44-49,105-108`). | #205 (closed) | P3 |
| Consumer SDK typed views | Split: register/typed-view the 4 #293-shipped types + extend existing `TOOL_COMPLETED` view now; `WORKFLOW_NODE_*` typed columns wait on #207 | #211 (v4 body) | P0 |

## Priority ranking

- **P0** — #206 (producer) + #211 (consumer SDK) in parallel. No shared code path; both unblocked.
- **P1** — #207 with pause-aware drain. Implementation downstream of P0 in customer urgency, unblocks `WORKFLOW_NODE_*` columns in #211.
- **P2** — #208. Matters only when an agent's `agent_state` exceeds the inline limit.
- **P3** — #205 closed as superseded; `source_event_id` join becomes the documented recipe in #211 / `SDK.md` (#220).

Concurrency: P0 items can be staffed in parallel (producer vs SDK). #207 design discussion can start in parallel; implementation lands after P0.

## Non-goals

- Do not reopen the #293 producer-only cut in this tracker unless a defect is found in the shipped fields.
- Do not block the customer's mid-June base-table JSON workflow on typed SDK views.
- Do not promise historical backfill for pair-key-only rows unless a separate backfill/update design is explicitly added.

## References

- #190 — full ADK 2.0 tracking issue.
- #293 — mid-June minimum producer cut.
- #206 — pause registry read-after-write (v4 body: soft-miss algorithm).
- #207 — workflow node boundary derivation (v4 body: event-observation + invocation-segment + ancestor propagation).
- #208 — oversized checkpoint offload (v3 body: canonical naming).
- #205 — OTel correlation (closed as superseded).
- #211 — SDK event-type / typed-view surface audit (v4 body: split "now" vs "#207 follow-up").
- #199 — full long-running `TOOL_PAUSED` / `TOOL_COMPLETED` contract; #293 covers the pair-key subset.
- Review threads (v1 → v4): see the comment thread on this issue.

---

Historical design-option text (pre-convergence)

Spawned from the post-#293 review / cutover discussion.

#### Context

#293 is the mid-June **producer-only minimum cut** for the customer's ADK 2.0 observability ask. After #293 lands in `adk-python`, the BQAA producer logs the customer-requested Event / EventActions fields that are direct producer copies or simple derived rows:

- `event.long_running_tool_ids` → `TOOL_PAUSED` + non-HITL long-running `TOOL_COMPLETED` pair keys.
- `event.node_info` → `attributes.adk.node.{path, run_id, parent_path}`.
- `actions.compaction` → `EVENT_COMPACTION`.
- `actions.transfer_to_agent` → `AGENT_TRANSFER`.
- `actions.end_of_agent` / `actions.agent_state` → `AGENT_STATE_CHECKPOINT` inline payload.
- `actions.route` / UI / rewind → `attributes.adk.{route, render_ui_widgets, rewind_before_invocation_id}`.

This issue tracks the remaining work that was **intentionally deferred** because it is not just copying fields from ADK `Event` into BQAA rows. Each item needs a design decision, a consumer-SDK contract, or both.

#### Remaining Work — original option-list framing

##### 1. Dedicated workflow boundary events

**Goal:** emit / expose `WORKFLOW_NODE_STARTING` and `WORKFLOW_NODE_COMPLETED`.

**Original options considered:** Event-observation derivation vs OTel-span consumption vs new ADK plugin lifecycle hook.

→ **Selected:** event-observation derivation, invocation-segment scoped (see #207 v4 body).

##### 2. Pause registry and `pause_orphan` semantics

**Original options considered:** in-process cache + BQ fallback / in-session ADK history reconstruction / settling-time grace window.

→ **Selected:** hybrid soft-miss algorithm via `session_service.get_session(num_recent_events=None)` + BQ settling read (see #206 v4 body).

##### 3. Oversized `AGENT_STATE_CHECKPOINT` GCS offload

**Original questions:** size threshold env var; URI placement; checksum placement; inline replacement; helper reuse.

→ **Selected:** canonical naming with dedicated structured-state helper (see #208 v3 body).

##### 4. OTel correlation / span id strategy

**Original options considered:** best-effort `otel_span_id`, `source_event_id` join, both.

→ **Selected:** `source_event_id` join only; #205 closed as superseded.

##### 5. Consumer SDK typed views and query surfaces

**Original options considered:** register types and extend `TOOL_COMPLETED` view vs document base-table reads.

→ **Selected:** register all six types + extend `TOOL_COMPLETED` view + carve out `WORKFLOW_NODE_*` typed columns as #207 follow-up (see #211 v4 body).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.