GoogleCloudPlatform / GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK
[Dashboard] Track dashboard usage via a query-attributable marker (read-path analog of the plugin's Write API trace_id)
- Dominant language
- Python
- Stars
- 47
- Forks
- 21
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 33
Description
Part of #404.
## Request
We have no signal on whether a published dashboard copy is actually used —
how often it's opened, by how many people, or what it costs its owner. The
only cost evidence today is point-in-time bytes-billed measurements (#401,
#377 section H). This issue adds **user-side usage tracking for the
dashboard's read path**, modeled on how the BQAA write path already solves
attribution.
## The pattern to reuse: the plugin's Write API `trace_id`
The ADK BQAA plugin stamps every Storage Write API append with a
client-identity trace ID
([`bigquery_agent_analytics_plugin.py:2592-2601` at `74e7167`](https://github.com/google/adk-python/blob/74e7167d13a75b8eaab6e30a7af39e8c55315fc0/src/google/adk/plugins/bigquery_agent_analytics_plugin.py#L2592-L2601),
class at [L4134](https://github.com/google/adk-python/blob/74e7167d13a75b8eaab6e30a7af39e8c55315fc0/src/google/adk/plugins/bigquery_agent_analytics_plugin.py#L4134)):
```python
req = bq_storage_types.AppendRowsRequest(
write_stream=self.write_stream,
trace_id=f"{trace_id_prefix}/{__version__}", # e.g. google-adk-bq-logger/1.x
)
```
Every write is attributable to the producer and its version, without any
side channel. The dashboard's **read path has no equivalent marker**: its
jobs run with the viewer's credentials in the viewer's billing project, and
nothing distinguishes them from any other query against the same table.
## Proposed mechanism (read-path analog)
- [ ] **Version-stamped marker in the template query.** Add a stable,
distinctive marker to `sql/events_v1.template.sql` — e.g. a leading
comment `/* bqaa-dashboard/ */` — so every job
the dashboard issues carries it in the job's query text.
**Verify first** that Looker Studio preserves SQL comments in the
jobs it issues (check `INFORMATION_SCHEMA.JOBS.query` against a live
copy); if comments are stripped, fall back to a no-op literal the
optimizer keeps (to be prototyped). The marker must not touch the
sentinel bindings (`sqlReplace` list unchanged) and gets pinned by
the existing generated-artifact drift tests.
- [ ] **Corroborating signal, no template change needed:** Looker Studio
stamps its jobs with the `requestor: looker_studio` job label —
usable today, but it cannot distinguish this dashboard from any
other Looker Studio report on the same table, which is why the
marker is the primary key and the label the cross-check. Verify the
label's presence/name against a live copy rather than assuming it.
- [ ] **A documented usage query** (docs + an oracle-style SQL file) over
`region-.INFORMATION_SCHEMA.JOBS_BY_PROJECT`, filtered on the
marker (and/or label + referenced table): queries/day, distinct
principals (`user_email`), total/percentile `total_bytes_billed`,
error rate, cache-hit rate. This is the read-path counterpart of the
#401 bytes-billed measurement, repeatable by any installation owner.
- [ ] **Docs:** a "Measure your dashboard's usage and cost" section in the
dashboard README, linking #377 section H's operating guidance.
## Privacy boundary — explicit non-goals
- **No phone-home.** All usage data stays in the installation owner's own
project; nothing reports to Google or this repository. The configurator
page keeps its no-backend property (#398/#399/#400) untouched.
- **No new collection.** The mechanism only makes *already-existing* job
metadata attributable; it adds no fields about viewers beyond what
BigQuery job history already records in the owner's project.
- Repo-side adoption metrics (how many installations exist) are explicitly
out of scope for this issue; if ever wanted, that is a separate
opt-in discussion.
## Coordination
Changing the template's custom query requires an editor session and a
republish. #401 is currently the batch's only planned republish — decide
whether this marker rides the same session (one republish event, per the
#404 plan's sequencing preference) or waits for the next scheduled one.
Either way the full #404 publication gate applies (all nine
`live_template_verification` methods, the 3×3 protocol, propagation probe,
attestation re-dating last).
## Acceptance criteria
- [ ] An installation owner can answer, with one documented query in their
own project: how often their copy was queried, by how many distinct
principals, at what bytes billed, over the last N days.
- [ ] The marker survives `sqlReplace` hydration unchanged and is pinned
by the generated-SQL drift tests.
- [ ] Comment-survival (or fallback-literal) behavior is verified against
a live copy and recorded here before the template changes.
- [ ] No data leaves the user's project; the configurator remains
backend-free; the docs state both explicitly.
- [ ] If the template changes, the republish passes the full #404
publication gate.
## Related
- #404 — tracking issue (this is new backlog, not in the current
execution ledger).
- #401 — republish coordination and the bytes-billed measurement this
generalizes.
- #377 section H — query-cost operating guidance.
- google/adk-python `bigquery_agent_analytics_plugin.py` — the write-path
precedent (pinned links above).
Contributor guide
Research direction
First verify against a live copy whether Looker Studio preserves SQL comments and emits the stated job label. Then inspect sql/events_v1.template.sql, the sqlReplace hydration and generated-SQL drift tests, plus the dashboard README and existing live_template_verification methods. Done means a documented JOBS_BY_PROJECT query reports usage and cost, privacy boundaries remain explicit, and any template change passes the full #404 publication gate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python, sql
- Domain
- analytics, databases, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100