GoogleCloudPlatform / GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK
Simple customer journey: admin deploys once, IT enrolls a Claude Code cohort, agent telemetry lands in BigQuery
- Dominant language
- Python
- Stars
- 47
- Forks
- 21
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 33
Description
## What this is
The **simple version** of the OTLP receiver customer journey: one page, main path only. The executable pilot contract lives in #363 — this issue is the story you tell first.
**In one sentence:** an enterprise admin uses one guided CLI workflow to deploy the telemetry pipeline in a dedicated company GCP project; IT enrolls an approved Claude Code pilot cohort through MDM, users need no local setup, and baseline usage, estimated-cost, latency, and reliability telemetry flows into BigQuery for SQL and existing BI tools.
---
## The journey (5 steps)
**Persona: Priya**, platform admin at a company rolling out AI coding agents. Leadership keeps asking: *"Who's using these tools? What do they cost? Are they reliable?"* Today the answers are **fragmented** — some in vendor analytics, some in billing, some in internal systems — and none of it joins up.
### 1. Deploy — one guided CLI workflow, about an hour
Priya runs the `bqaa-otel bootstrap` workflow against a dedicated GCP project: it prints a full plan first, and nothing runs until she re-invokes it with `--execute`. It creates everything — the receiver endpoint on Cloud Run, the pipeline behind it, and the BigQuery tables — all inside **her company's project, not a vendor's**. (Until the packaged release #349 ships, this runs from an approved repo checkout.)
> **Target packaged experience after #349** — today's approved pilot still uses #363's checkout path.
```bash
# Preview the deployment; nothing changes
bqaa-otel bootstrap \
--project acme-agent-pilot \
--dataset agent_analytics \
--region us-central1
# Apply the reviewed plan
bqaa-otel bootstrap \
--project acme-agent-pilot \
--dataset agent_analytics \
--region us-central1 \
--execute
```
The target workflow deploys the receiver and pipeline, creates the governed BigQuery dataset, and generates the Claude Code starter configuration.
### 2. IT completes and distributes one config
Bootstrap generates a **starter config** for Claude Code. IT completes it (access token, approved privacy and identity settings), validates it, and **distributes it as endpoint-managed settings through the approved MDM workflow; Priya approves the enrolled cohort.**
### 3. Enroll the pilot cohort — zero local setup for users
Enrolled users perform no configuration. **Telemetry begins after each enrolled client's next full restart** (OpenTelemetry settings apply on restart), then flows to the receiver and lands in BigQuery within minutes. **Baseline excludes prompt, code, and tool-output content, but includes approved employee identity (reported via OAuth, reconciled to the roster — not cryptographically verified) and operational metadata** — it is content-free, not anonymous. *(Codex support is future-state, pending its identity and attribution work — see #363 Gate B2.)*
#### What one prompt looks like in BigQuery
These synthetic rows show three linked events from one Claude Code prompt. They share the same employee identity and `prompt_id`; intermediate events are omitted for readability.
| Time | Event | Employee identity | Prompt ID | Operational data | Content at baseline |
|---|---|---|---|---|---|
| 14:32:01 | `user_prompt` | `user_01BW…` | `7f4a…` | Prompt length: 184 characters | Prompt redacted |
| 14:32:04 | `api_request` | `user_01BW…` | `7f4a…` | Model: Sonnet; estimated cost: $0.018; latency: 2,840 ms | No request/response body |
| 14:32:06 | `tool_result` | `user_01BW…` | `7f4a…` | Tool: Read; success: true; duration: 120 ms | Tool input/output not captured |
**Readable schema:** `time` says when the event occurred; `event` says what Claude Code did; `employee identity` is the OAuth account ID used for roster matching; `prompt_id` joins activity caused by one prompt; `operational data` contains event-specific model, cost, latency, or tool status; and `content at baseline` makes the privacy boundary visible.
This is a PM-friendly normalized view derived from native OTel log attributes, not the physical BigQuery storage schema. Under the pilot baseline, prompt and assistant text, raw API bodies, and tool input/output remain redacted or absent.
### 4. Verify — pipeline, then a real client
Run the smoke check to prove the pipeline end-to-end with a synthetic signal, then confirm one redacted row arrives from an enrolled client after its restart. (Exact commands and acceptance checks live in #363.)
### 5. Analyze and decide — it's just BigQuery now
The data lands in the governed BigQuery dataset. Priya's data team points SQL, dashboards, and the BQAA SDK at it: usage by team (reported identity joined to the roster), **estimated** usage cost trends (provider billing stays authoritative), latency, error rates. **The signals are queryable on day one and become decision-grade after #363's coverage and observation thresholds pass** — then Priya takes the evidence to leadership to target cost or reliability improvements and the expansion decision.
> **Illustrative query result after the coverage and observation thresholds pass.** Values are synthetic; estimated cost is not the provider bill.
| Team | Active / enrolled | Sessions | Est. cost | p95 API latency | API error rate |
|---|---:|---:|---:|---:|---:|
| Payments | 38 / 50 | 214 | $186 | 2.8 s | 1.4% |
| Search | 21 / 30 | 121 | $94 | 4.7 s | 4.9% |
| Infrastructure | 28 / 35 | 176 | $132 | 3.1 s | 0.8% |
Priya can see that Search has reasonable adoption but materially worse latency and errors, giving leadership a concrete reliability action — not merely another usage dashboard. This is a derived BigQuery query result, not the physical logging-table schema, and it uses baseline operational metadata without prompts, code, or tool-output content.
---
## Why enterprises want their agent telemetry in BigQuery
1. **Fresher, deeper signals — then BigQuery does the rest.** Vendor analytics covers adoption and spend summaries. OTLP adds **near-real-time, event-level latency and failure signals**. BigQuery then provides governed history, internal joins (rosters, cost centers, billing), and provider-independent analysis — in your GCP project under your IAM, retention, and region.
2. **One governed BigQuery dataset and common analytics surface.** Every enrolled agent lands in the same dataset, which extends to more products as they become supported.
3. **Answer the leadership questions with SQL.** Adoption by team and **estimated** spend by cost center (via joins from reported identity to your roster/directory — provider billing remains authoritative for actuals), latency and failure trends — standard BigQuery, no new tools to learn. Existing BI (Looker, dashboards) just works.
4. **Privacy is controlled at the source.** The baseline tier excludes prompt, code, and tool-output content (it does include approved employee identity and operational metadata). Richer tiers are explicit, separately approved opt-ins the company controls.
5. **It compounds.** Baseline telemetry makes usage, estimated cost, latency, and reliability queryable on day one; the evidence becomes decision-grade after #363's coverage and observation thresholds pass. With the separately approved richer-data journey, the rest of the BQAA toolkit unlocks — agent evaluation, LLM-as-judge scoring, replay-based curation — turning "we collect telemetry" into "we measure and improve agent quality."
---
## Guardrails (the short version)
Run the pilot in a **fresh, dedicated GCP project**, use OAuth login (so usage maps to real seats), and keep the baseline privacy tier until security review approves more. **Trust boundary, stated plainly: the receiver uses public Cloud Run ingress authenticated by the shared ingestion credential, and it stores what authenticated managed clients send — it does not independently enforce the privacy tier. #363 defines ingress bounds, monitoring, rotation, and containment.** The full pilot contract — gates, rotation, incident procedures, acceptance metrics — is #363; its linked versioned runbook will be the operating manual. This issue is the front door.
**Related:** #363 (pilot contract), #316 (receiver), #349 (packaged release).
Contributor guide
Research direction
Start with the executable pilot contract in #363, then review the related receiver work in #316 and packaged release in #349. The bqaa-otel bootstrap workflow is the named entry point, but this issue describes a customer journey rather than a bounded code change; completion criteria therefore remain in the linked pilot contract.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python, shell
- Domain
- cli, cloud, data, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100