GoogleCloudPlatform / GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK
Admin agent skill: let Claude Code / Codex drive the full rollout — install → deployed, validated, finalized BQ logging dataset
- Dominant language
- Python
- Stars
- 47
- Forks
- 21
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 33
Description
## Why
The #316 → #344 arc produced a complete admin stack and a rehearsed demo — and, as a byproduct, a recorded transcript of every mistake an agent can make deploying it (nonexistent gcloud subcommands, org-policy blocks, per-product landing false-greens, enum drift, unreliable shutdown flushes, unverifiable teardowns). An **agent skill** distills that operational knowledge so an enterprise admin can point their own Claude Code or Codex at a GCP project and get from `pipx install` to a finalized, validated telemetry dataset — with the judgment points and safety rails we learned live built in.
Depends on **#349** (`bootstrap --preflight` and `bqaa-otel teardown` must exist as subcommands first).
## Core design principle: skill orchestrates, CLI executes
No deployment logic in the skill's bash. Mechanics live in `bqaa-otel` (preflight/bootstrap/verify/teardown, all live-proven); the skill is the **conversational conductor** — what to run next, how to interpret failure, when to stop and ask the human. This keeps the skill thin and product-agnostic: the same content drives Claude Code and Codex because the commands do the heavy lifting, and the skill versions with the CLI it drives (`requires bqaa-otel >= 0.2.0` — skill/CLI drift is the failure mode to design against).
## Phase structure (mirrors the proven live sequence)
```
0. Install pipx install bigquery-agent-analytics-tracing==X → version check
1. Interview project / dataset / region / signals / privacy tier
2. Preflight bqaa-otel bootstrap --preflight → interpret via failure map
3. Deploy show plan (dry-run) → explicit human confirm → --execute
4. Enable distribute the two generated artifacts (the branchy part)
5. Validate real session → per-product landing gates → verify --smoke
6. Finalize retention policy, analyst IAM, saved views, DLQ/freshness alerts
7. Handoff inventory recorded for teardown + evidence summary doc
```
Two phases carry most of the authoring effort:
- **Phase 2 — the failure→fix map.** Learned live: org policy blocks `allUsers` → name the constraint and the ask ("get a policy exception for `iam.allowedPolicyMemberDomains`"); missing `iam.serviceAccounts.actAs` → the exact role to request; billing invisible → who to contact. The agent should emit the *remediation sentence*, not the raw error. Seed content: the demo OPERATOR.md troubleshooting table (built from real failures).
- **Phase 4 — where agents improvise dangerously without guidance.** Branches by product and channel: Claude managed settings via admin console (**human-only paste** — the skill must say "I cannot do this step; here is exactly what to paste") vs MDM; Codex `config.toml` via managed dotfiles with the embedded-token warning. This phase is mostly instructions *to the human*, and the skill is explicit about that boundary.
## Safety rails (each one is a live scar)
- **The token never enters the chat.** Secret Manager references only; the skill forbids echoing generated Codex configs (they contain the literal bearer token).
- **Per-product landing gates, never aggregate.** "It works" where only one product landed is the exact false-green caught in #345 review round 3.
- **Verification is existence checks, not exit codes** (the rule the #346 teardown review forced — probe failures must never masquerade as "gone").
- **Privacy is a one-way door with a human key.** The skill never constructs the replay acknowledgement flag itself; the human must type it. Mirrors the CLI's exit-2 gate at the conversational layer.
- **Dry-run before anything destructive; teardown only via the recorded inventory** with allowlist-guarded names.
## Distribution
- **Claude Code**: ship the skill inside the plugin tarball that `release-tracing.yml` already builds — versioned with the CLI.
- **Codex**: emit the same content as an AGENTS.md playbook. Candidate mechanism: `bqaa-otel init-skill --agent claude-code|codex` so one source generates both — symmetric with how the product already generates per-product config artifacts.
## Acceptance: the demo is the eval
The #344 rehearsal is the golden transcript, so the skill's acceptance test is concrete:
> Fresh GCP project, clean machine, an agent given only the skill and a project ID completes phases 0–7 **without human help except the designed-in confirmation points** (deploy confirm, artifact paste, privacy-tier choice).
- [ ] Skill source authored (phases, failure→fix map, safety rails), with the skill/CLI version pin
- [ ] `init-skill` (or equivalent) emits Claude Code skill + Codex AGENTS.md from one source
- [ ] Wired into the release plugin tarball
- [ ] Eval run recorded: agent-driven rollout on a fresh dataset, graded against the rehearsal transcript — evidence posted here
- [ ] Negative tests: agent correctly *refuses* to echo tokens, to enable replay without the human ack, and to tear down without a dry-run shown
Contributor guide
Research direction
Start with the demo OPERATOR.md troubleshooting table and the existing release-tracing.yml plugin packaging. Confirm the #349 subcommands and their versioning, then design init-skill to emit the Claude Code skill and Codex AGENTS.md from one source. Done means the rehearsal-based eval and negative tests pass, with evidence recorded for the rollout and safety rails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- cli, cloud, devops, observability, release, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100