Customer user journey: OTLP receiver — enterprise pilot → hardened rollout (for PM review)

## Summary

End-to-end **customer user journey** for the OTel-native OTLP receiver (`deploy/otlp_receiver` — implemented on `main` under #316; **release pending #349**), for PM review. **Two artifacts with separate change control:** this issue is the **stable PM contract** — supported shape, gates, risks, ownership, metrics, product priorities; the command blocks, SQL, IAM manifests, and rotation/teardown procedures embedded in Stages 2–6 are the **draft seed of the versioned operational runbook** (a required, linked A1 artifact). Once the runbook exists, operational corrections version there **without PM re-approval**; only changes to the contract sections above return here.

> ⚠️ **Resource-ownership warning (P0, contained):** preflight treats a pre-existing dataset as healthy, bootstrap converges onto existing resources, and **teardown recursively deletes the entire dataset** (`bq rm -r -f`) and, in source mode, the whole `bqaa` Artifact Registry repo. **The pilot MUST run in a fresh, dedicated GCP project with a new dataset**, and the executable absence check below must pass (archived) before A0 closes. Product fix: B1 row 6.

> **Who decides what — and what THIS issue asks for:**
> - **The decision requested here is Template/pilot-shape approval** by the **BQAA PM**: the journey template, the supported pilot shape (Claude Code-only, OAuth/gateway identity, MDM-only, dedicated project), and the B-lane priorities/staffing.
> - **Gate A0 closure is a different, later decision** by a **named pilot customer** (Leadership + Admin + Security + IT + Claude org Owner), once they select the project, auth mode, data inventory, thresholds, and residual posture. This issue cannot close A0 — no customer is named in it.

> **Validated against:** repo pin **`96a079f`** — behavioral validation was performed at `d7b41e3`; the delta to `96a079f` is lock-only ([PR #367](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/pull/367) replaced the **PyPI-yanked `build==1.5.1`** in the hash-locked receiver image build; do NOT deploy from `d7b41e3`). HEAD observed at `08d1dd8` on 2026-07-15 (intervening changes release-path-only); **the normative pin is `96a079f` until explicit revalidation** — do not chase HEAD. **Source-build and digest evidence must be re-captured at the new pin.** · Claude Code docs as of 2026-07-15 · fleet **≥ 2.1.193**, exact canary binary pinned at A1 (expected 2.1.203 per #349) · codex-cli 0.142.5 (#317) · review date 2026-07-15.

> **Revision 18 (surgical — closes the review cycle per the [v17 POV](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4988689633)):** pre-cutover rollback now repins the heartbeat to N; the constraints recipe excludes the root distribution (path-portable); the server-managed no-merge rule qualified for the `policyHelper`/lock-key exceptions (both excluded from this pilot). **This document is submitted for PM template/pilot-shape approval.**
> **Revision 17** — applied the [independent v16 review](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4988380477) (1 P0 + 11 P1 + 2 P2): **P0 fixed — the secret-lifecycle decision now precedes teardown as mutually exclusive branches**; the rotation protocol unified (heartbeat first, one resume point, rollback repins both, pre-registered outage threshold); continuity measured against an independently observable expected-active denominator; coverage renamed reported-identity coverage with the trusted-client assumption at A0; preserved backlog exported outside teardown scope; Stage 4 consumes the full contract (main-subscription health, continuity, matched-row mismatch); constraints generation made exact; heartbeat IAM bootstrap authority named; the raw-telemetry runbook promoted to an A1 artifact; the PM contract/runbook change-control split declared; a PM decision table with recommendations added; propagation fixes; support contract + project retirement added. (Prior revision: [v15→v16](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4988200431) — initial numeric-secret pinning step added (bootstrap deploys `:latest`), **DTS recognized as the third writer** in quiescence, the heartbeat marker moved to a log attribute that survives the projection crosswalk, pipeline availability separated from heartbeat-excluded real-source freshness with subwindow continuity, deployer IAM corrected (`run.jobs.setIamPolicy`), constraints made reproducible (repo-owned constraints = B1 row 18), **Gate C now requires B1a+B1b+B1c**, rotation rollback invariants added, retention exceptions bounded, and the no-go evidence archive inventoried. (Prior: [v14→v15](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4986526167) — the stale direct-provider clause deleted from Success Metric 2, the constraints file made a required tested artifact (with an explicit unconstrained-path risk acceptance), **sink-write quiescence inserted as a real step in the no-go sequence** (the v14 disposition had overstated this), projection staleness checked over every eligible key (sampling only as a named canary), the freshness predicate and half-open window defined precisely, one authoritative rotation sequence (Scheduler pause + heartbeat version update merged in), retention *enforcement* added to Gate C, the digest assertion tightened to `sha256:` + 64 hex, deployer Cloud Run/Scheduler roles listed, and the PM ask split into three checkboxes. (Review trail: [v13→v14](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4986379337) · [v12→v13](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4986278135) · [v11→v12](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4986112960) · [v10→v11](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4985719575) · [v9→v10](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4985494999) · [v1→v2](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4977857914) · [v2→v3](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4978063069) · [v3→v4](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4978303099) · [v4→v5](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4978593561) · [v5→v6](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4978718251) · [v6→v7](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4978864613) · [v7→v8](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4979026453) · [v8→v9](https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK/issues/363#issuecomment-4983763249).)

**Related:** #316 (open), #349 (open), #317 (closed), #324 (closed).

---

## Readiness gates

| Gate | Unlocks | Closed by |
|---|---|---|
| **A0 — Deploy pilot infra** | Stage 2 deployment | Pilot-customer Leadership |
| **A1 — Onboard pilot users** | **Cohort distribution** and Stages 4–5 with real users (Stage 3 up to and including synthetic validation is pre-A1) | Pilot-customer Admin + Security |
| **B1a — Publication accuracy** | Honest customer-facing docs | BQAA maintainers; PM approves |
| **B1b — Released install path** | Wheel/pinned-image install (#349 + lifecycle smoke) | BQAA maintainers |
| **B1c — Product reliability** | Safe teardown/rotation/ingress at scale | BQAA maintainers |
| **B2 — Codex eligibility** | Codex joins the pilot | Named BQAA owner |
| **C — Broad rollout** | Expansion beyond the pilot | Requires **B1a + B1b + B1c** (and B2 if Codex in scope) — a broad rollout on a released package with known-inaccurate customer-facing instructions (B1a) is not permitted; the only exception is a named customer using the approved runbook, recorded explicitly. Customer lane: Admin + Security + IT; BQAA lane: named maintainer |

### Gate A0 checklist (pilot customer)
- [ ] **Fresh dedicated GCP project + new dataset**, with the **executable absence check** below run and its all-absent output archived (preflight passes on existing resources — do not rely on it):
```bash
#!/usr/bin/env bash
# Fail-closed absence check: succeeds ONLY on a recognized not-found for every
# resource. Existing resources, auth/permission/location errors all fail.
# (No `set -e`: expected absence makes these commands return nonzero.)
#
# PRE-STEP (bounded mutation, acknowledged): a truly fresh project has the read
# APIs disabled, so the probes would fail as API errors. Enable them first:
# gcloud services enable run.googleapis.com artifactregistry.googleapis.com \
# pubsub.googleapis.com secretmanager.googleapis.com bigquery.googleapis.com \
# bigquerydatatransfer.googleapis.com cloudscheduler.googleapis.com \
# iam.googleapis.com --project=my-pilot-proj # iam: the SA-list probe fails without it
# The runbook must record this script tested against real clean-project output
# for every resource class (gcloud wording varies by service and version).
set -u
P=my-pilot-proj; R=us-central1; DS=agent_analytics; FAIL=0
must_be_absent() { # $1 label; $2... command
local label=$1; shift
local out; out=$("$@" 2>&1)
if [ $? -eq 0 ]; then echo "FAIL — EXISTS: $label"; FAIL=1
elif grep -qiE 'not.?found|does not exist|NOT_FOUND|cannot find|was not found|no such' <<<"$out"; then echo "ok — absent: $label"
else echo "FAIL — ERROR checking $label: $out"; FAIL=1; fi
}
must_be_absent dataset bq show --dataset "$P:$DS"
must_be_absent ar-repo gcloud artifacts repositories describe bqaa --location="$R" --project="$P"
must_be_absent run-receiver gcloud run services describe bqaa-otlp-receiver --region="$R" --project="$P"
must_be_absent run-consumer gcloud run services describe bqaa-otlp-consumer --region="$R" --project="$P"
must_be_absent topic-main gcloud pubsub topics describe bqaa-otlp --project="$P"
must_be_absent topic-dlq gcloud pubsub topics describe bqaa-otlp-dlq --project="$P"
must_be_absent sub-main gcloud pubsub subscriptions describe bqaa-otlp-sub --project="$P"
must_be_absent sub-dlq gcloud pubsub subscriptions describe bqaa-otlp-dlq-sub --project="$P"
must_be_absent secret gcloud secrets describe bqaa-otlp-token --project="$P"
sas=$(gcloud iam service-accounts list --project="$P" --filter="email~bqaa" --format='value(email)') \
|| { echo "FAIL — ERROR listing service accounts"; FAIL=1; }
[ -z "${sas:-}" ] || { echo "FAIL — EXISTS: service accounts: $sas"; FAIL=1; }
dts=$(bq ls --transfer_config --transfer_location=US --project_id="$P" 2>&1) \
|| { echo "FAIL — ERROR listing DTS configs: $dts"; FAIL=1; }
grep -q bqaa_agent_events_otlp_merge <<<"$dts" && { echo "FAIL — EXISTS: DTS merge config"; FAIL=1; }
[ "$FAIL" -eq 0 ] && echo "ALL ABSENT — archive this output" || echo "ABSENCE CHECK FAILED"
exit "$FAIL"
```
A `--require-absent` preflight mode is B1c product work (row 11). Until this script (or that mode) gates A0, the fresh dedicated project is the real containment control and the archived output is the evidence.
- [ ] Authentication mode selected; **identity & coverage contract** fixed (Stage 1).
- [ ] **Security-approved data inventory covering every storage surface** (Stage 1) — including Secret Manager.
- [ ] **Decision contract** written (Stage 1) — including the transport-DLQ windowed definition and the time-to-first-real-row anchor/target.
- [ ] Interim posture approved **with ingress bounding**: public Cloud Run ingress + shared bearer token; the WSGI app reads the full request body before auth, Cloud Run accepts 32 MiB HTTP/1 bodies, and **bootstrap deploys with no scaling bounds** (defaults allow up to ~100 instances, briefly exceedable) — the Stage 2 bounding commands are mandatory **before the endpoint or token is disclosed to anyone**. Deploy-time bounds are B1c row 12.
- [ ] **Pilot data lifetime bounded**: max lifetime, dated teardown deadline, owner; deletion verification distinguishes logical deletion from final expiry (BigQuery time-travel + ~7-day fail-safe; Pub/Sub unacked retention up to 7 days; **Secret Manager delayed-destruction settings**).
- [ ] **Heartbeat actor decided** (Stage 1 contract: Scheduler → Cloud Run job → Secret Manager) and its surfaces added to the inventory.
- [ ] **Cohort-bounding plan** acknowledged by IT and the Claude org Owner (Stage 3 rules).

### Gate A1 checklist (pilot customer — after Stages 2–3)
- [ ] Receiver deployed; `inventory.json`, source SHA, Python version, and **both services' revision image digests** recorded (commands in Stage 2).
- [ ] **Effective-configuration proof on dedicated synthetic-validation machines/identities** (never cohort machines — cohort distribution is what A1 *unlocks*):
1. Distribute the **approved final artifact** (Stage 3 assembly) → **fully restart Claude Code**.
2. `/status` → intended managed source active.
3. `claude doctor` → payload validates.
4. Machine-side check → applied payload matches the approved artifact by **canonical semantic comparison against the policy hash** (token sentinel — see Stage 3).
5. **Telemetry canary** on the exact pinned Claude version → endpoint, identity attribute, selected signals, redaction proven in BigQuery.
- [ ] **Availability heartbeat provisioned first** per the Stage 1 contract (owner, scheduler, credential store, marker, exclusion rule verified) — the rotation rehearsal requires it.
- [ ] **Maintenance-rotation rehearsal** — the ONE Stage 3 sequence, heartbeat pause/repin included (old token proven 401, per-rotation artifact receipt + re-canary). The emergency branches reuse this sequence with the hard-stop/immediate variations.
- [ ] **Security-approved raw-telemetry access runbook** (an A1 artifact, not future B1a docs): authorized readers, audit logging, redacted-evidence rules, prohibited exports, replay approval, incident escalation, verified disposal — Stage 4 invokes it whenever a DLQ is naturally nonzero.
- [ ] **Acceptance runbook linked from this issue and dry-run.**
- [ ] **Conditional — if delayed secret destruction was selected at A0:** the preserve-secret **manual teardown checklist is linked, reviewed, and dry-run before cohort distribution** (a no-go can happen immediately after onboarding; "written before use" is too late).

**Product eligibility:** Claude Code-only; Codex arrives with Gate B2.

### Gate B1 evidence table (a lane closes only when its rows have Owner, Milestone, Issue/PR, Evidence, Status filled)

| # | Condition | Lane | Owner | Milestone | Issue/PR | Evidence | Status |
|---|---|---|---|---|---|---|---|
| 1 | README/CLI help: templates-not-ready-to-distribute; source-specific privacy matrix | B1a | TBD | TBD | — | — | open |
| 2 | `pipx install bigquery-agent-analytics-tracing` labeled **pending release** (not on PyPI) | B1a | TBD | TBD | — | — | open |
| 3 | Windows MDM guidance off `C:\ProgramData` → `C:\Program Files\ClaudeCode\` / `HKLM` policy | B1a | TBD | TBD | — | — | open |
| 4 | Generated artifacts explicitly set the five content controls + `OTEL_METRICS_INCLUDE_ACCOUNT_UUID` | B1a | TBD | TBD | — | — | open |
| 5 | Bootstrap binds a **numeric secret version** instead of `:latest` | B1c | TBD | TBD | — | — | open |
| 6 | **Retry-safe ownership-aware teardown**: durable deployment ID, pre-mutation journal, atomic per-resource ownership updates, refusal to delete unknown ownership, partial-failure/resume tests (“created by this deployment across retries,” not just “this run” — inventory is currently written only after success, so a failed attempt strands or mis-adopts resources) | B1c | TBD | TBD | — | — | open |
| 7 | Raw-DLQ access/retention controls documented | B1a | TBD | TBD | — | — | open |
| 8 | #349: wheel on PyPI + digest-pinned image + install/upgrade/teardown lifecycle smoke | B1b | TBD | TBD | #349 | — | open |
| 9 | Receiver rejects oversized requests **before reading the body** | B1c | TBD | TBD | — | — | open |
| 10 | **Dual-token receiver support**: overlap protocol, compatibility test, lifecycle evidence (unlocks continuous rotation — until then all rotations are outage-window maintenance) | B1c | TBD | TBD | — | — | open |
| 11 | Preflight `--require-absent` mode (fail if any managed name exists) | B1c | TBD | TBD | — | — | open |
| 12 | Bootstrap applies deploy-time scaling/concurrency bounds (no window of unbounded public service) | B1c | TBD | TBD | — | — | open |
| 13 | Source-build image digest printed and inventoried (source mode builds a mutable `:latest`; the digest assertion only runs for digest-qualified inputs) | B1c | TBD | TBD | — | — | open |
| 14 | Teardown `--preserve-secret` option (today it reads `inventory["secret"]` unconditionally; delayed-destruction no-go requires manual teardown) | B1c | TBD | TBD | — | — | open |
| 15 | Inventory/teardown coverage for heartbeat resources (Cloud Run job, Scheduler trigger, SAs, invoker + secret-access grants) | B1c | TBD | TBD | — | — | open |
| 16 | Source-ingest provenance column in `agent_events_otlp` (staleness of MERGE-updated rows is currently only detectable by row-hash comparison) | B1c | TBD | TBD | — | — | open |
| 17 | Teardown ordering: stop consumer + pause DTS + drain/dispose backlog before dataset deletion | B1c | TBD | TBD | — | — | open |
| 18 | Repo-owned, hash-locked runtime **and build** constraints for the admin CLI (reviewed hash-checking install workflow) | B1c | TBD | TBD | — | — | open |

---

## Current state vs released state

| | Today (repo checkout) | Released install path (after **B1b**) |
|---|---|---|
| Install | pinned-SHA checkout + venv install (below) + `--build-from-source` | `pip/pipx install`; embedded pinned image |

**Checkout installation (fail-fast; run as a script, not pasted lines):**
```bash
set -euo pipefail
git clone https://github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK.git
cd BigQuery-Agent-Analytics-SDK
git checkout 96a079feae3898a02176a3e53df3e0f530fe6673 # includes PR #367's yanked-build lock repair
[ -z "$(git status --porcelain)" ] || { echo "DIRTY TREE — aborting" >&2; exit 1; }
python3 -m venv .venv && source .venv/bin/activate
# REQUIRED artifact: pilot-constraints.txt — EXACT generation procedure (until repo-owned
# hash-locked constraints ship, B1 row 18; alternatively the customer may make row 18 an A1
# prerequisite at A0). On a clean reference machine:
# python3 -m venv ref && source ref/bin/activate
# python3 -m pip install pip==
# pip install ./producers # NON-editable: constraints cannot contain editable reqs
# pip freeze --exclude-editable \
# --exclude bigquery-agent-analytics-tracing \
# > pilot-constraints.txt
# (the root distribution must be excluded: freezing it emits a file:/// constraint
# that only resolves from the identical checkout path — the file constrains DEPENDENCIES)
# Review, record sha256 + the python/pip versions in the change ticket. At install time, assert
# the same python/pip versions before proceeding. RESIDUAL RISK (explicitly accepted at A0):
# a constraints file bounds versions but is NOT a hash lock, and does not govern the isolated
# hatchling build environment.
echo " $HOME/pilot-constraints.txt" | shasum -a 256 -c - # asserts the checksum, not just readability
pip install -e producers/ -c "$HOME/pilot-constraints.txt"
# Explicitly-unconstrained alternative — requires recorded customer risk acceptance:
# pip install -e producers/ # live resolution; the later pip freeze is evidence, not a pin
command -v bqaa-otel && bqaa-otel --help >/dev/null # console script IS installed by the editable install
pip freeze > "$HOME/pilot-lock.txt" # OUTSIDE the checkout (Cloud Build uploads the working dir as-is); this freeze is DEPLOYMENT EVIDENCE, not the pin
python3 --version && git rev-parse HEAD # record both in the change ticket
```
Bootstrap (incl. `--preflight`) additionally requires `--build-from-source` (image resolved before preflight). `verify`/`teardown` need only the install. **Re-run the clean-tree check immediately before `--execute`.**

#349/B1b is an install milestone; enterprise rollout is reached after Gate C.

---

## Persona

**Priya, Enterprise AI Platform Admin** adopting Claude Code (Codex later); org holds **MDM authority**. Supporting cast: **IT/endpoint-management owner**, **Dana** (data science), **Sam** (security/compliance).

---

## Stage 1 — Discover, decide, approve deployment (Gate A0)

**Identity & coverage contract — pre-register the authentication mode; identity follows:**

| Auth mode | Seat identity | Coverage denominator | Notes |
|---|---|---|---|
| Anthropic org OAuth | `user.account_id` (Admin-API-tagged) | Admin API roster | set `OTEL_METRICS_INCLUDE_ACCOUNT_UUID=true` explicitly, test in the A1 canary; `user.email` may be emitted — inventory it |
| Claude apps gateway | `user.id` = IdP subject (+ `user.email`, `user.groups`) | IdP directory | `user.groups` inventoried |
| Direct API key / Bedrock / Vertex / Foundry | none emitted | — | **NOT eligible for this pilot.** A random per-installation `user.id` cannot prove an installation belongs to an approved seat, so roster coverage, unknown-identity reporting, and the "approved-cohort identity" success contract are all unsatisfiable. A direct-provider pilot needs its own separate contract built on non-roster **"active installation signals"** — out of scope here |

**OAuth or gateway identity is mandatory for this pilot** — the success contract (roster coverage, unknown/unapproved reporting, approved-cohort first-row) is only satisfiable with `user.account_id` or an IdP subject.

**Coverage = observed approved identities ∩ roster ÷ roster** — and it is **reported-identity coverage, not verified attribution**: the receiver authenticates endpoints only by the shared bearer token and then trusts submitted OTLP attributes, so a compromised cohort machine could forge an approved identity. The **trusted-client assumption is recorded at A0**, reconciled against MDM receipts, and no claim of cryptographically verified seat attribution is made. Unknown/unapproved identities reported separately. `user.id` outside the gateway is a random per-installation ID.

**Sensitive-data inventory (Security approves every storage surface):**

| Surface | Contents | Notes |
|---|---|---|
| Native BigQuery tables + `agent_events_otlp` | telemetry incl. identity attributes | time-travel + ~7-day fail-safe outlive logical deletion |
| Application DLQ (`otlp_dead_letter`) | replayable `raw_b64` | operator-only; explicit retention |
| Transport DLQ (Pub/Sub) | failing envelope | operator-only; explicit retention |
| Main Pub/Sub topic/subscription | envelopes carry the decoded record + resource attributes | unacked retention up to 7 days default; IAM/retention deliberate |
| **Secret Manager (`bqaa-otlp-token`)** | the bearer token, **plus every rotation's added versions** | access grants enumerated (incl. the heartbeat job SA); rotation evidence retained; **lifecycle split — see Stage 3/5**: rotation = disable + scheduled *version* destruction; full teardown = `gcloud secrets delete` (immediate, removes the secret and ALL versions — delayed destruction applies only to version-level destroy); if policy requires delayed destruction at no-go, the secret must be preserved through a version-level disable/destroy procedure instead of teardown's delete |
| Secret-bearing config artifacts | token in MDM payloads and endpoint copies | distribution + revocation procedure |
| Identity fields | `user.account_id`, `user.email`, `user.groups`, `user.id`/`session.id` | classified; access + retention defined |
| **No-go evidence archive** | preserved incident/decision evidence (may contain raw telemetry excerpts) | location, contents, access policy, retention, and deletion proof defined at A0 — or the archive is limited to aggregate/redacted evidence only |

**Named health metrics (Stage 4 consumes these verbatim):**

| Metric | Executable definition (pre-registered window) |
|---|---|
| Application dead-letter record failure rate | distinct DLQ `idempotency_key`s ÷ distinct (native ∪ DLQ) keys |
| **Transport-DLQ (windowed)** | **max backlog over the window + oldest-message age + delivery/disposition counts** — not just instantaneous backlog, so a transient incident that drains before acceptance still shows |
| Unattributed-row rate | rows missing the per-signal required-key list ÷ total rows |
| Missing-metrics rate | active identities with logs but no metric flushes ÷ **eligible** identities (session activity ≥ one export interval, default 60 s, + grace) |
| **Pipeline availability** | the **provisioned heartbeat** (below) lands in the sink in **every subwindow** (e.g. each 15-min slice); any missing subwindow = outage |
| **Real-source freshness/continuity (heartbeat-EXCLUDED)** | heartbeat-inclusive `MAX(ingest_time)` can stay green while every Claude client is dark — so real-source metrics exclude heartbeat rows and split into **three distinct measures** (the roster proves *eligibility*, not expected activity in any subwindow — one active seat must not keep aggregate continuity green): ① **onboarding coverage** (one-time: distinct approved identities ever observed ÷ roster); ② **active-seat continuity** per subwindow against ③ an **independently observable expected-active denominator** — scheduled guided sessions during the acceptance window and/or endpoint-side activity evidence — never roster prediction. Expected-active > 0 with zero matching rows in a subwindow = failure (pipeline outage excluded by the heartbeat); expected-active = 0 ⇒ N/A |
| **Main-subscription health** | backlog message count, oldest-unacked age, and acknowledgement throughput on `bqaa-otlp-sub` (Cloud Monitoring), thresholds set in Gate A0 |
| **Native freshness (per logical signal; pipeline time; window-scoped)** | `SELECT TIMESTAMP_DIFF(@window_end, MAX(ingest_time), SECOND) FROM WHERE ingest_time >= @window_start AND ingest_time < @window_end AND ` per expected physical table (**half-open interval** so adjacent archived windows never double-count a boundary row; the exclusion predicate removes smoke, canary, and synthetic-validation identities; **heartbeat rows are excluded from real-source freshness but included in pipeline-availability and reliability metrics** — see the two rows above) — logs → `otel_logs`; metrics → `otel_metric_sum` (Claude emits **counters**; other point-kind tables = **N/A**); traces → `otel_spans` when enabled. **Whole-table MAX is wrong**: pre-A1 canary/synthetic rows make tables nonempty, so an idle window would read stale instead of N/A. **Zero rows in-window → N/A; the heartbeat decides idle vs outage** (for the heartbeat's own table, zero in-window rows = failure). `bqaa_metrics` exposes event time, not `ingest_time` — NOT the freshness source |
| Projection backlog/freshness | Two checks. **Missing rows:** `SELECT COUNT(*), TIMESTAMP_DIFF(CURRENT_TIMESTAMP(), MIN(l.ingest_time), SECOND) FROM otel_logs_dedup l LEFT JOIN agent_events_otlp p USING (idempotency_key) WHERE p.idempotency_key IS NULL AND l.ingest_time < TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL MINUTE)` — count 0 ⇒ no backlog (lag NULL = healthy). **Stale matched rows** (the MERGE updates existing keys, so an anti-join can't see a replayed/repaired row whose projection wasn't refreshed, and the projection has no source `ingest_time`): **pilot acceptance runs a mismatch COUNT over every eligible key older than the MERGE grace period** (deterministic hash of expected crosswalk row vs target row; runbook SQL). If sampling is ever substituted, it must be named a canary with pre-registered sample design/confidence and the residual blind spot stated. Source-ingest provenance in the projection is B1c row 16 |
| User/seat coverage | per the identity contract |

**Availability heartbeat contract (do NOT schedule `verify --smoke` for this — it emits a synthetic `claude_code.user_prompt` under default product attribution and runs the projection `MERGE` directly, which would contaminate acceptance denominators and mask a broken scheduled query):**
- A **POST-only** OTLP log record whose marker **survives the projection**: the crosswalk keeps `log_attributes` but drops general resource attributes and `service.name`, so the marker must be a **log attribute** (`bqaa.heartbeat=true`) plus a reserved event name — a resource-attribute-only marker would make heartbeat exclusion impossible in standalone `agent_events_otlp` queries. (A resource attribute may be set additionally for native-table queries.)
- **Concrete actor, decided at A0 (the receiver accepts only its exact bearer token, so Scheduler cannot authenticate to it directly, and a bearer-in-header Scheduler job would itself become a secret-bearing surface):** Cloud Scheduler triggers a minimal **Cloud Run job** (`bqaa-heartbeat`) via the Cloud Run Jobs API `…jobs/bqaa-heartbeat:run` endpoint using **OAuth** (`--oauth-service-account-email` — jobs are invoked with OAuth access tokens, not OIDC). The job's runtime SA reads `bqaa-otlp-token` from Secret Manager at run time and POSTs the marked record; the bearer is **never persistently stored outside Secret Manager** — transient exposure exists in job memory, the TLS `Authorization` header, and receiver memory, and is inventoried as such.
- **IAM/prereqs:** Scheduler caller SA with `roles/run.invoker` on the job; separate job runtime SA with `roles/secretmanager.secretAccessor` (least-privilege default; may be combined by explicit decision); `cloudscheduler.googleapis.com` enabled; the job image **digest-pinned** with a named executable.
- **Ownership & cleanup (product inventory/teardown do NOT cover jobs or Scheduler triggers):** the heartbeat **must ship** as its own small deployment manifest with created-before state and an explicit cleanup script — job, trigger, both SAs, invoker grant, and secret-access grant (a required, linked A1 artifact alongside the runbook). **No-go containment pauses the Scheduler trigger first**, before touching the receiver or secret. (Product coverage is B1c row 15.)
- **Secret-version protocol (defined, not "reads the current version"):** the job is **pinned to the same numeric secret version as the receiver**. Rotation follows **the single Stage 3 maintenance sequence — the trigger resumes only after fleet redistribution is verified**, not after receiver cutover. (`latest`/aliases lack numeric-version consistency guarantees and would flip the job's token before the receiver redeploys.)
- **Deployment prereqs in the manifest:** the deployer's `roles/iam.serviceAccountUser` (`actAs`) on both SAs; **Cloud Run job rights including IAM binding** — `roles/run.developer` covers job lifecycle but **not** `run.jobs.setIamPolicy`, which granting/cleaning the `run.invoker` binding requires, so use job-scoped `roles/run.admin` or a narrow custom role adding `run.jobs.setIamPolicy` — **and name the bootstrap authority for the circularity that a job-scoped binding cannot exist before the job does**: either the deployer temporarily holds a project-level custom role covering job creation + `run.jobs.setIamPolicy` (downscoped after manifest execution), or a separate IAM administrator grants the job policy after creation; **Scheduler job create/update/delete** rights (`roles/cloudscheduler.admin` or custom `cloudscheduler.jobs.*`); and verification that the Cloud Scheduler service agent role exists in the project.
- **Initial numeric pinning (bootstrap deploys `bqaa-otlp-token:latest` — the numeric protocol does not exist until this step runs):** resolve the current version N (`gcloud secrets versions list`), redeploy the receiver referencing `:N` explicitly, assert the revision↔version mapping, **then** create the heartbeat job pinned to N — and only then rehearse the N→N+1 rotation at A1.
- **Inventoried at A0:** all of the above.
- **Excluded from value and coverage metrics** by the marker; **included in reliability/failure metrics** — `otlp_dead_letter` exposes no resource attributes and transport-DLQ backlog/age are aggregates, so heartbeat failures cannot be filtered out there, and shouldn't be: exposing pipeline failures is the heartbeat's job. Observed **read-only** in the sink (no MERGE trigger).
- DTS/`MERGE` execution monitored **separately** (transfer-run history), so the heartbeat cannot hide a dead scheduled query.
- Owner, cadence, and retention/cleanup defined in the acceptance runbook; provisioned before A1 closes.

**Privacy: enforced, not detected** (fleet ≥ 2.1.193; exact canary version pinned at A1):
```
OTEL_LOG_USER_PROMPTS=0
OTEL_LOG_ASSISTANT_RESPONSES=0 # falls back to the prompts setting when unset — set it anyway
OTEL_LOG_TOOL_DETAILS=0 # security-audit tier: 1
OTEL_LOG_TOOL_CONTENT=0
OTEL_LOG_RAW_API_BODIES=0
```
The privacy canary (Stage 4) is drift **detection**; these flags are the enforcement.

**Privacy matrix** (receiver stores what each authenticated source sends; no tier enforcement): baseline = logs+metrics with the zeros above (Codex, when eligible via B2: `tool_result` carries an output snippet — version-sensitive, no documented cap); security-audit = `OTEL_LOG_TOOL_DETAILS=1` (not a distinct Codex tier); replay = prompt capture with explicit acknowledgement, raw bodies deferred (never offered for Codex). Traces ≠ replay. Codex `[analytics]` is independent of `[otel]`.

**Decision contract:** usage/cost/safety questions each with a **status-quo comparator**; numeric thresholds for every metric above; pre-registered window + minimum denominators; the **falsifiable decision-use record** (pre-registered action or confidence change per outcome, incl. no-value → no-go); baseline "safety visibility" definition; and the **time-to-first-real-row anchor**: measured from the **first MDM cohort-assignment timestamp** (captured in the post-A1 handoff) to the first row bearing an **approved-cohort identity**, excluding smoke, heartbeat, canary, and synthetic-validation identities; pre-registered target.

**Exit:** Gate A0 closed by pilot-customer Leadership.

## Stage 2 — Install, preflight, plan, deploy (~30–60 min)

After the pinned-SHA installation and the archived absence check:

```bash
claude --version # separately, on validation machines — do NOT use --check-products (it also demands the codex CLI)

bqaa-otel bootstrap --project my-pilot-proj --dataset agent_analytics \
--region us-central1 --source claude-code --signals logs,metrics \
--privacy baseline --build-from-source --preflight

bqaa-otel bootstrap --project my-pilot-proj --dataset agent_analytics \
--region us-central1 --source claude-code --signals logs,metrics \
--privacy baseline --resource-attributes department=eng,cost_center=1234,env=pilot \
--build-from-source # dry-run plan → change ticket

[ -z "$(git status --porcelain)" ] || { echo "DIRTY TREE — aborting" >&2; exit 1; } # re-check before execute
bqaa-otel bootstrap --project my-pilot-proj --dataset agent_analytics \
--region us-central1 --source claude-code --signals logs,metrics \
--privacy baseline --resource-attributes department=eng,cost_center=1234,env=pilot \
--build-from-source --execute # writes inventory.json at the end
```

**Immediately after `--execute`, and before the endpoint or token is disclosed to anyone:**
```bash
# Bound the public service (bootstrap deploys with NO scaling bounds):
gcloud run services update bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj \
--max-instances=3 --concurrency=80
gcloud run services update bqaa-otlp-consumer --region=us-central1 --project=my-pilot-proj \
--max-instances=3
# Verify with fail-fast ASSERTIONS (not prints) and archive the output:
set -euo pipefail
rx_max=$(gcloud run services describe bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj \
--format='value(spec.template.metadata.annotations["autoscaling.knative.dev/maxScale"])')
rx_cc=$(gcloud run services describe bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj \
--format='value(spec.template.spec.containerConcurrency)')
cs_max=$(gcloud run services describe bqaa-otlp-consumer --region=us-central1 --project=my-pilot-proj \
--format='value(spec.template.metadata.annotations["autoscaling.knative.dev/maxScale"])')
[ "$rx_max" = "3" ] && [ "$rx_cc" = "80" ] && [ "$cs_max" = "3" ] \
|| { echo "BOUNDS NOT APPLIED (receiver max=$rx_max cc=$rx_cc consumer max=$cs_max)"; exit 1; }
# Record AND ASSERT the actually-deployed image digests (source mode builds a mutable
# :latest and bootstrap does not record its digest — B1c row 13). An empty value fails:
for s in bqaa-otlp-receiver bqaa-otlp-consumer; do
rev=$(gcloud run services describe $s --region=us-central1 --project=my-pilot-proj --format='value(status.latestReadyRevisionName)')
dig=$(gcloud run revisions describe "$rev" --region=us-central1 --project=my-pilot-proj --format='value(status.imageDigest)')
grep -qE '(^|@)sha256:[0-9a-f]{64}$' <<<"$dig" && echo "$s $rev $dig" || { echo "BAD DIGEST for $s ('$dig')"; exit 1; }
done
```
Plus budget alerts on the project. Attach `~/pilot-lock.txt`, the SHA/Python record, digests, and the non-CLI decision contract to the change ticket.

**Failure branch — partial deploy:** re-run the same convergent command. Stranded resources stay contained in the dedicated project (retry-safe ownership is B1c row 6).

## Stage 3 — Bounded distribution and credential procedures (closes Gate A1)

**Cohort bounding — the pilot is MDM-only:** server-managed settings are org-wide (no per-group targeting); a server payload delivering **any keys at all** disables endpoint-managed settings entirely (no merging — with two documented exceptions that do not apply here: a configured `policyHelper` preempts all managed sources, and a small set of cross-source lock keys is honored from any admin source; **this pilot excludes `policyHelper`**); cached server settings persist until a successful refresh. The pilot therefore uses cohort-targeted **MDM** with a server payload delivering **no keys**, proven by A1's `/status` step after a refresh. **A whole-org server-managed pilot is out of scope for this journey**: `/status` proves the active source but not per-key provenance, and there is no supported semantic payload readback — so the A1 "applied payload matches the policy hash" proof is not executable for server-managed delivery. Running that variant would need its own proof contract (admin-console artifact hash + audit record + `/status` + canary) and would add the vendor-side settings storage and client caches to the token-bearing inventory. **Windows:** `C:\Program Files\ClaudeCode\managed-settings.json` or `HKLM\SOFTWARE\Policies\ClaudeCode` (generated `C:\ProgramData` guidance is stale — B1 row 3).

**Final-artifact assembly (mandatory, before any distribution):**
1. Start from the generated template (it omits privacy controls and the identity flag — B1 row 4).
2. Add the identity flag and the five privacy controls per the A0 contract.
3. Validate + canonicalize the **policy artifact with the token replaced by a fixed sentinel**; record the **Security-approved policy hash** — it is **stable across token rotations**.
4. Inject the real token to produce the distribution artifact; record a **restricted per-rotation artifact receipt** (secret version ↔ artifact version ↔ distribution record).
5. Distribute to **synthetic validation users only** (pre-A1). **Cohort distribution happens strictly after A1 closes, as a controlled handoff** — the sequence is: synthetic validation passes + heartbeat provisioned + runbook complete → **close A1 (approval recorded)** → **first MDM cohort assignment issued** → **that assignment's timestamp captured** as the metric anchor → distribution proceeds. **Every rotation updates the receipt and re-runs the canary; the policy hash needs re-approval only when policy content changes.**

**Token handling:**
1. **Maintenance rotation — the ONE authoritative sequence (explicit outage window; single-token receiver — dual-token is B1 row 10); the A1 rehearsal exercises exactly this:** announce window → **pause the heartbeat Scheduler trigger** → create version N+1 → update **both** the receiver secret reference **and** the heartbeat job's pinned numeric version to N+1 (numeric pinning fixes `:latest` early pickup; it does **not** create overlap) → force + verify a new receiver revision → record revision↔version → rebuild the distribution artifact + receipt → redistribute → verify new works/old 401s → **resume the Scheduler trigger** and confirm a heartbeat lands → **disable version N and schedule its version-level destruction** per retention policy (do **not** delete the secret — that removes all versions immediately).
**Rollback invariants:** version N stays enabled until fleet reconciliation completes — **as rollback material only**: once the single-token receiver is on N+1, N-configured clients are broken regardless (keeping N enabled restores nothing by itself). Failure *before* receiver cutover → abort, **repin and verify the heartbeat job on N** (the sequence may already have moved it to N+1), then resume the trigger on N — **no fleet cutover occurred** (the heartbeat repin is a change, hence not "nothing changed"). Failure *after* cutover but *before* distribution → either roll **both the receiver and the heartbeat job** back to N (and resume on N) or accept the outage and push distribution forward — a recorded decision against a **pre-registered outage/rollback threshold** (maximum tolerated dark-fleet fraction × duration), not a default. Failure *during* partial fleet distribution → hold both artifact receipts, reconcile per machine, decide roll-forward vs roll-back at the threshold, and do not disable N until every endpoint is proven on N+1. Every abort path either restores the heartbeat trigger or intentionally keeps it paused under an open incident.
2. **Containment has two distinct levels — receiver-zero is NOT sink quiescence.** Scaling the receiver to zero stops **new accepted requests (ingress containment)** only; **three writers** can still change the dataset: the consumer draining the main Pub/Sub subscription, **and the DTS scheduled `MERGE` rewriting `agent_events_otlp` every ~15 minutes**. **Sink-write quiescence** additionally requires draining/stopping the consumer with backlog disposition **and pausing the transfer config** (full sequence in the Stage 5 no-go steps). Evidence capture and teardown decisions come only after quiescence (product teardown also deletes the dataset before the consumer/subscriptions — B1c row 17).
3. **Emergency branches — containment commands are pre-tested:**
```bash
# Hard stop (new RECEIVER ACCEPTANCE halts — the consumer keeps draining Pub/Sub into
# BigQuery until the quiescence step; automation that re-enables services must be suspended first):
gcloud run services update bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj --scaling=0
# PROOF of manual-zero — manual scaling lives in SERVICE-LEVEL metadata.annotations,
# not the revision template. Assert, don't just print:
mode=$(gcloud run services describe bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj \
--format='value(metadata.annotations["run.googleapis.com/scalingMode"])')
cnt=$(gcloud run services describe bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj \
--format='value(metadata.annotations["run.googleapis.com/manualInstanceCount"])')
[ "$mode" = "manual" ] && [ "$cnt" = "0" ] || { echo "NOT CONTAINED (mode=$mode count=$cnt)"; exit 1; }
# ... rotate secret, redeploy, redistribute ...
# Restore, assert EXACT automatic mode (fail-closed: an empty/failed read must not pass),
# then REAPPLY AND RE-ASSERT the approved bounds:
set -euo pipefail
gcloud run services update bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj --scaling=auto
mode=$(gcloud run services describe bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj \
--format='value(metadata.annotations["run.googleapis.com/scalingMode"])')
cnt=$(gcloud run services describe bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj \
--format='value(metadata.annotations["run.googleapis.com/manualInstanceCount"])')
[ "$mode" = "automatic" ] && [ -z "$cnt" ] || { echo "NOT AUTOMATIC (mode='$mode' count='$cnt')"; exit 1; }
gcloud run services update bqaa-otlp-receiver --region=us-central1 --project=my-pilot-proj --max-instances=3 --concurrency=80
# ...then re-run the Stage 2 bounds assertion block.
```
Or **immediate rotation** (same without the stop, accepting the outage); then exposure-window integrity inspection and incident record.

**Exit:** Gate A1 closed by pilot-customer Admin + Security; MDM deployment rate reported separately.

## Stage 4 — Acceptance contract (~30 min + observation window)

Executed via the linked acceptance runbook.

**4.1 Pipeline smoke** (advisory only — nonzero dead letters → `WARN` + exit 0). **Smoke has no single synthetic marker**: logs/spans carry `bqaa.run_id`, metrics use generated `bqaa_e2e_*` names, and failed smoke records cannot be filtered from either DLQ. Therefore: **the health observation window starts only after smoke completes**; successful smoke rows are excluded with explicit per-table predicates (`bqaa.run_id` for logs/spans, the `bqaa_e2e_` name prefix for metrics); and **any smoke failure fails Stage 4 outright** — no filtering around it:
```bash
BQAA_OTLP_TOKEN= bqaa-otel verify \
--endpoint --project my-pilot-proj --dataset agent_analytics \
--signals logs,metrics --smoke
```

**4.2 Acceptance checks — Stage 1 metrics verbatim; threshold breach fails the pilot regardless of CLI exit status:**
- [ ] Application dead-letter record failure rate within threshold.
- [ ] **Transport-DLQ windowed metrics** (max backlog, oldest-message age, disposition counts) within thresholds.
- [ ] Every selected real-source signal from real sessions (spans only when traces selected).
- [ ] Missing-metrics rate within threshold (export-interval eligibility rule).
- [ ] Coverage per the identity contract ≥ threshold (incl. identity-flag presence test).
- [ ] Unattributed-row rate within threshold.
- [ ] **Availability heartbeat** present in every subwindow; **DTS transfer-run history healthy** (checked separately — the heartbeat cannot prove the scheduled MERGE).
- [ ] **Real-source continuity** (heartbeat-excluded, per subwindow, against the expected-active denominator).
- [ ] **Main-subscription health** (backlog count, oldest-unacked age, ack throughput) within thresholds.
- [ ] **Privacy canary (drift detection):** scripted marker sessions; expected fields present, forbidden content absent across native tables, projection, selected spans/metrics. DLQ content testing (both surfaces) is a **Gate C fault-injection control test**; naturally-nonzero DLQ rows during the window are inspected under the raw-telemetry procedure as part of disposition.
- [ ] Native freshness **per logical signal**, projection backlog (count + oldest-lag), **and the full matched-row mismatch count** within thresholds.

**Window-reset rule:** any conformance-affecting change restarts the window and denominators for affected metrics.

## Stage 5 — Analyze; go/no-go against the Stage 1 contract

Scoring: pre-registered thresholds, denominators, status-quo comparator; the decision-use record closes exactly as pre-registered (no-value → the pre-registered no-go). Analytics scope: SDK auto-detects `agent_events`/`agent_events_v2` only — explicitly select `agent_events_otlp`; some projected fields null; latency/token detail in native tables.

**No-go branch — safe order, complete cleanup:**
1. **Pause the heartbeat Scheduler trigger and prove it paused** (per the heartbeat cleanup manifest).
2. **Admin contains the receiver ingress** (pre-tested `--scaling=0` command above, or token rotation) — new acceptance stops; **the consumer keeps draining into BigQuery**.
3. **Achieve sink-write quiescence — THREE writers, an explicit executed decision for each:**
- **Consumer:** either drain the main subscription and prove the backlog empty over a **defined quiet interval** (backlog metrics can lag by minutes; for push subscriptions also check the outstanding-message metric, not a single backlog observation), waiting for settled writes; **or** scale the consumer to zero, prove it, then preserve or dispose the remaining backlog (recorded disposition).
- **DTS (the scheduled `MERGE` — it keeps rewriting `agent_events_otlp` every ~15 min regardless of the services):** run/await one final successful MERGE, **pause/disable the transfer config**, wait for any active run to finish.
- **Proof:** native and projected table row counts/hashes stable across the quiet interval. Only now is the dataset's content final.
4. **Preserve incident/decision evidence** (now that writes are settled).
5. **IT removes endpoint configuration** and proves removal.
6. **Run and verify heartbeat-manifest cleanup** — job, trigger, both SAs, invoker and secret-access grants all gone.
7. **Secret-lifecycle decision — BEFORE any teardown; the branches are mutually exclusive:**
- **(a) Immediate deletion acceptable** (default for the bounded pilot) → proceed to step 8a, product teardown.
- **(b) Delayed destruction required** → **never invoke product teardown** (it runs `gcloud secrets delete` unconditionally — the secret and all versions die immediately; it reads `inventory["secret"]` with no skip, and `--dataset-only` preserves the whole pipeline, not just the secret). Proceed to step 8b, the manual preserve-secret teardown. A `--preserve-secret` product option is B1c row 14.
Also decide the **preserved backlog** here if step 3 chose preservation: product teardown deletes the main subscription, so first **export the backlog to a retained destination outside teardown scope and verify the export** — or forbid teardown until the backlog has its final recorded disposition.
8. **Teardown — exactly one branch:**
**(8a) Product teardown** (safe only in the dedicated project; deletes the dataset before the consumer/subscriptions — B1c row 17 — which step 3 has made safe):
```bash
bqaa-otel teardown --project my-pilot-proj --dataset agent_analytics --inventory inventory.json # dry-run preview
bqaa-otel teardown --project my-pilot-proj --dataset agent_analytics --inventory inventory.json --confirm # destructive, after review
```
**(8b) Manual preserve-secret teardown** (checklist **written and reviewed before use** — a conditional A1 artifact): delete every resource except the secret; run version-level disable → scheduled destroy; delete the secret only after the windows expire.
9. **Security verifies every inventory surface** — both DLQs, the main subscription (or its exported backlog), heartbeat cleanup, endpoint artifact copies, the evidence archive — and records post-deletion retention states (time-travel/fail-safe windows running vs. finally expired).
10. **Final control — retire the project:** once every approved retention window has expired, **delete the dedicated project** — the strongest guarantee against stranded IAM grants, service agents, build artifacts, logs, and partially-inventoried resources.

## Stage 6a — Gate C hardening (before *any* expansion beyond the approved pilot)

**Customer readiness** (Admin + Security + IT): private ingress/gateway; MDM-only distribution; **retention *enforcement*, not just the 6b detection row** — BigQuery partition expiration on every generated table (DDL currently sets none), an application-DLQ row-deletion procedure, Pub/Sub retention settings aligned to policy, secret-version destruction schedules, **or** a **bounded exception** in place of enforcement — never open-ended: named owner, expiry date, compensating alert, maximum exposure window, and a renewal decision at expiry; alerting per 6b; **staged expansion with stop conditions** (pilot → ~100 → ~200, each cohort gated on the health metrics at the new load) or a target-load test with safety factor; **DLQ fault-injection control tests (both surfaces, with cleanup)**; pseudonymous endpoint correlation if endpoint-level closure is required — **evaluate Claude Code's MDM-only `policyHelper`** as a concrete correlation path at the pinned fleet floor (it can derive managed settings from device posture/identity, at the cost of a new executable, cache, and security proof surface).

**BQAA readiness** (named maintainer): B1c rows shipped — **dual-token rotation (row 10)**, numeric-version bootstrap (5), retry-safe ownership teardown (6), pre-read size rejection (9), `--require-absent` preflight (11), deploy-time bounds (12), source-digest inventory (13), `--preserve-secret` (14), heartbeat coverage (15), projection provenance (16), teardown ordering (17), hash-locked constraints (18); lifecycle commitments — plus an explicit **customer/BQAA support contract**: escalation ownership, severity definitions, response expectations, compatibility/deprecation policy, and security-incident responsibility (Stage 6b currently assigns nearly every operational loop to the customer).

## Stage 6b — Perpetual operations

| Surface | Owner / cadence | Alert | Closing check |
|---|---|---|---|
| Receiver/pipeline | Admin / on alert | 5xx, consumer failures | `verify --smoke` green |
| Application DLQ | Admin / daily + alert | record failure rate over threshold | rate within threshold + recorded disposition per nonzero result |
| Transport DLQ | Admin / alert | windowed backlog/age over threshold | drained + disposition recorded |
| Projection backlog | Admin / alert | count/oldest-lag > threshold | backlog SQL clean |
| **DTS / scheduled MERGE** | Admin / alert | failed or missing transfer runs | healthy run history |
| Availability heartbeat | Admin / continuous | heartbeat missing | heartbeat restored + gap investigated |
| Privacy drift | Security / scheduled + on version change | canary failure | canary re-run clean |
| Endpoint config failure | IT / on alert | identity dark/non-conforming | two proofs: ① machine policy compliance (restart + `/status` + policy hash), ② identity emitting conforming telemetry (correlation work is Gate C) |
| **Retention enforcement** | Security / **cadence per policy — at least as frequent as the shortest A0-approved lifetime** (monthly cannot enforce a shorter-than-monthly lifetime) | any surface past its approved lifetime | evidence that BigQuery partitions (generated DDL sets **no expiration**), the transport-DLQ **subscription resource** (configured never to expire — distinct from its **messages'** 7-day retention setting), main-subscription retention, and disabled secret versions are all within the A0-approved lifetimes; breaches escalate to the teardown-deadline owner |

Flow: contain → diagnose → recover → re-verify with the surface's closing check. Plus rotation drills (with artifact receipts); monthly compatibility smoke with a **BQAA maintainer obligation** for vendor drift (fleet floor ≥ 2.1.193; canary version pinned).

**Follow-on journey (separate doc):** `replay` — Claude Code only, opt-in; raw bodies deferred; flywheel not yet end-to-end.

---

## Journey at a glance

| Step | Time | Owner | Exit | Biggest risk |
|---|---|---|---|---|
| 1. **Gate A0** | Day 0–2 | Pilot customer | dedicated project + archived absence check + contract + inventory + bounded-ingress plan | teardown vs. shared project (P0); **wrong auth-mode identity or denominator** |
| 2. Deploy | ~1 hr | Admin | pinned SHA deployed; bounds applied + verified pre-disclosure; digests recorded | unbounded public service window; dirty tree |
| 3. Distribution + **Gate A1** | Days 1–3 | IT + Admin + Security | assembled-artifact proof chain (policy hash + receipt) + canary on pinned version + heartbeat provisioned | cached server payload overriding MDM |
| 4. Acceptance | 30 min + window | Admin | all metrics within threshold incl. heartbeat + DTS history | trusting advisory CLI exit / empty-DLQ "proof" |
| 5. Analyze; go/no-go | per window | Data science + Leadership | decision-use record closed as pre-registered | post-hoc rationalization; slow no-go containment |
| **B1a/B1b/B1c/B2** | product work | BQAA | evidence-table rows closed | docs / install / reliability / Codex blocked |
| 6a. **Gate C** | before expansion | Customer + BQAA | both checklists + staged evidence + **B1a+B1b+B1c** | scaling from checkout past evidence |
| 6b. Operate | perpetual | Admin (+ BQAA obligation) | per-surface closing checks green | raw-DLQ exposure; silent drift |

---

## Success metrics

1. **Time-to-first-real-row:** from the **first MDM cohort-assignment timestamp** (post-A1 handoff) to the first row bearing an **approved-cohort identity**; excludes smoke, heartbeat, canary, and synthetic-validation identities; pre-registered target.
1b. **End-to-end time-to-value:** from **pilot kickoff** (A0 work begins) to the Stage 5 decision — including customer labor and blocked time across A0/A1, so the headline metric cannot hide the days of setup effort preceding cohort assignment.
2. **Coverage per the identity contract** — roster coverage under OAuth or gateway identity (the only modes this pilot supports); MDM deployment rate reported separately.
3. **Decision value:** decision-use record closed as pre-registered, beating the comparator by the pre-registered margin.

## Open questions

**For the BQAA PM — three separate dispositions, with recommendations:**

| # | Decision | Recommendation | Staffing | Dependencies | Target | If deferred |
|---|---|---|---|---|---|---|
| 1 | Approve the journey template + supported pilot shape (Claude Code-only, OAuth/gateway, MDM-only, dedicated project) | **Approve** | none (decision only) | — | now | no pilot customer can be engaged; review cycle continues without an anchor |
| 2 | Staff the B1 evidence table (**18 rows**) | Priority order: **row 6** (ownership-safe teardown — P0 root cause), **row 10** (dual-token — every rotation is an outage until then), **row 17** (teardown ordering), then B1a docs rows | ~1 eng for B1c reliability rows + tech-writer pass for B1a | row 8 depends on #349 | rows 6/10/17 within one milestone | pilot proceeds under documented mitigations; broad rollout (Gate C) stays blocked |
| 3 | Confirm B-lane ordering (B1a accuracy / B1b release / B1c reliability) | **B1c → B1a → B1b** (safety first, then docs honesty, then release) | as above | B1b = #349 | with #349's milestone | Gate C requires all three anyway; ordering only shifts interim risk |
**For the named pilot customer (a later, separate decision — not requested here):** authentication mode (OAuth or gateway only); data-inventory + residual-posture acceptance (incl. ingress cost exposure); A0 closure with the archived absence check.

Start with deploy/otlp_receiver and the Readiness gates, then review the Stage 1–6 contract and the linked operational-runbook requirements. Confirm the fresh-project absence check, release and reliability gates, ownership, metrics, and PM decision table. Done means the PM approves the pilot shape and the required evidence and runbook artifacts are clearly assigned.

Written by the indexing model from the issue text.