GoogleCloudPlatform / GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK
Looker Studio dashboard: UX/design/analytics improvement backlog
- Dominant language
- Python
- Stars
- 47
- Forks
- 21
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 33
Description
Improvement backlog for the published BQAA Looker Studio template ([report `5a3f85ef`](https://lookerstudio.google.com/reporting/5a3f85ef-fc9c-4730-8ef2-8ef9129ddb40), source under `dashboard/looker_studio/`, from #365 / #374 / #376).
Findings come from the parity manifest (`spec/chart_manifest.yaml` — geometry, styling, listener wiring extracted from the pinned block), the SQL contracts, and the configurator source. Row/column numbers below are the manifest's 24-column grid coordinates.
**Framing note for contributors:** the current layout, titles, and colors deliberately mirror the pinned community block for parity (M0 contract). Most items below therefore mean *intentionally diverging from block parity* (or upstreaming fixes to the block). Grouping items by decision weight so peers can pick things up independently.
## A. Bugs / low-risk fixes
- [ ] **LLM percentile row is out of order.** Latency page row 23 renders P50 (col 0), **P99** (col 6), **P75** (col 12), **P90** (col 18). The tool percentile row above it (row 10) is correctly P50→P75→P90→P99. Reorder the LLM row to match.
- [ ] **`Events By Agent` (Tool Usage page) is mislabeled.** Its query counts only `TOOL_COMPLETED` events, not all events. Rename to something like "Tool Completions by Agent" (or broaden the query if the original intent was all events).
- [ ] **Unlabeled/empty elements.** User Analytics has an empty BUTTON (row 0) and an empty TEXT element (row 1); Latency has an empty-label BUTTON (row 0). Label them, wire them to something useful (e.g. "Reset filters"), or remove them.
- [ ] **Section header mismatch.** Agent & Sessions opens with an h1 reading just "Agent" (row 0) above the *traces* section, then "Sessions" at row 15. The first header should describe the section (e.g. "Traces").
## B. Layout & alignment (24-col grid)
- [ ] **Inconsistent scorecard/trend split across sections.** Scorecard+trend pairs use w7+w17 (avg tool latency), w8+w16 (sessions, LLM calls, users, errors, avg LLM latency), and w9+w15 (traces, tokens). Pick one ratio and apply it everywhere.
- [ ] **Off-grid full-width bars.** `Top 5 Agents Split by Session Count` (col 1, w22) and `Top 5 Agents by LLM Calls` (col 1, w21) start at col 1 instead of col 0/w24 like every other full-width element — they render visually off-center.
- [ ] **Inconsistent heights for same-role tiles.** Top-5 bar charts vary h7/h8/h9; scorecards vary h5/h6/h7 with no semantic reason.
- [ ] **Inconsistent page openings.** Agent & Sessions, User Analytics, and Latency start content at row 2 under an h1; LLM Interactions, Token Consumption, Tool Usage, and Errors start at row 0 with no page header. Add headers everywhere or drop them everywhere.
## C. Visual design & color
- [ ] **No semantic color system.** The same metric changes color between charts: sessions is `#e8710a` (orange) in one chart and `#e52592` (pink) in another; tool errors appear as `#e8710a`, `#f9ab00`, and `#a50e0e` across three charts; tokens are `#1e8e3e` in one and `#e8710a` in another. Define one palette keyed by metric family (tokens, sessions, traces, latency, errors) and apply it consistently. Reserve red hues for error metrics only.
- [ ] **Legend position is `center` on all 37 charts** (including scorecards, where it's meaningless). Verify how this renders and standardize (top-right or top-left is conventional for time series).
- [ ] **Value labels are off on all charts** (`show_value_labels: false`). For top-5 bar charts, on-bar value labels would remove the need to hover.
- [ ] **Number formatting.** Token totals render as raw integers; consider compact notation (1.2M) on scorecards and axis labels, thousands separators in tooltips, and explicit `ms` unit suffixes on latency scorecards.
- [ ] **Report theme.** Evaluate a branded Looker Studio theme (typography scale for h1/h2 headers, muted gridlines, consistent scorecard font sizes) instead of the block's defaults. Dark theme variant if maintained cheaply.
## D. Naming & copy
- [ ] **Capitalization is inconsistent across tiles**: `P50 Llm Latency` ("Llm"), `Top 5 users with most Traces` (lowercase "users"), `Traces split by Agent` / `Token Usage split by Agent` (lowercase "split"), `Top 5 Agents By Errors` vs `Top 5 Tools by Errors` (By/by). Adopt title case everywhere.
- [ ] **Grammar**: `Total Traces Generation Over the Time` and `Total Tokens Consumption Over the Time` → "… Over Time"; `Top 5 users with most Tokens consumption` → "Top 5 Users by Token Consumption".
- [ ] **Unit style**: `Average LLM Latency (in ms)` vs `Average Tool Latency (ms)` — pick one form.
- [ ] **Ambiguity**: `Total Calls` (LLM Interactions) → "Total LLM Calls"; `Top 5 Users by Session` → "…by Sessions".
## E. Filters & interactivity
- [ ] **Tool Name filter affects exactly one chart** (`Average Tool Latency (ms)`). The other 11 performance charts — including the tool latency trend, all four tool percentiles, and both tool-error charts — ignore it. Wire it to every tool-scoped chart, or scope/label the control so the behavior isn't surprising.
- [ ] **Four usage charts ignore the User ID filter**, two of which are literally "Top 5 users" charts (`Top 5 users with most Traces`, `Top 5 users with most Tokens consumption`, plus `Token Usage split by Agent`, `Traces split by Agent`). Decide intentionally: wire them or document why not.
- [ ] **Session ID filter is missing** even though `session_id` is a first-class schema field and two pages are session-centric.
- [ ] **Model Version filter is missing** despite `model_version` existing in the 30-field schema — useful for comparing model rollouts.
- [ ] **Date window story is fragmented**: usage pages default to 14 days, performance pages to 7 days, and the report-level default range is rolling 365 days. Harmonize (or document) so users aren't surprised by cross-page differences.
## F. Analytics enhancements (post-parity)
- [ ] **Percentile scorecards → richer visual.** Eight separate P50/P75/P90/P99 scorecards could become two compact tables or bullet charts, freeing space for percentile *trend* lines (p50/p95 bands on the latency trend charts), which show regressions over time that static percentiles hide.
- [ ] **Previous-period comparison is inconsistent**: 6 scorecards have it (tokens, traces, sessions, calls, users, errors); average-latency and percentile scorecards don't. Add comparison uniformly where meaningful.
- [ ] **Error rate, not just error counts**: tool error % of invocations and LLM error % of calls are more actionable than absolute counts.
- [ ] **Cost/estimated-spend view**: token counts × configurable price table is a frequently requested agent-analytics view.
- [ ] **Hourly granularity** for short windows (trend charts are daily-only, so a 1-day view collapses to one point).
- [ ] **Inspector page polish**: link each chart's drill-through to the Inspector page with filters carried over, so "what are these 12 errors" is one click.
## G. Configurator page (`docs/`)
- [ ] **No favicon** and **no OpenGraph/Twitter meta tags** — links shared in chat/social render bare.
- [ ] **No dark mode** (no `prefers-color-scheme` in `styles.css`; single 820px breakpoint is the only media query).
- [ ] **Field-level validation hints**: errors are generic ("Enter a valid BigQuery project ID"); show the expected format inline (e.g. "6–30 chars, lowercase letters, digits, hyphens").
- [ ] **"Try the demo" path**: a one-click link that opens the template over the synthetic fixture would let evaluators see the dashboard before wiring their own data.
- [ ] **Post-create checklist UX**: the Viewer's-Credentials gate is documented in prose; consider a copyable checklist (or a short GIF) since it is the one step users must not skip.
## H. Operations & performance
- [ ] **Large-table guidance**: the single custom query is date-pruned but still scans the window per chart interaction. Document expected bytes-billed at common scales and recommend partition-expiration / BI Engine / rollup-table options for large installs.
- [ ] **Freshness indicator**: a "data through " scorecard so viewers know staleness at a glance.
- [ ] **Template ownership transfer** to a Google-managed account (tracked in `bindings/report_template.yaml` governance block; listed here for visibility).
## I. Accessibility
- [ ] Verify color contrast of series colors against the white theme (e.g. `#f9ab00` amber on white is likely below WCAG AA for thin lines) and check the palette for color-blind distinguishability (orange/pink pairs above are risky).
- [ ] Chart titles should not rely on color alone to distinguish tool vs LLM sections — the h1/h2 header fixes in section B help here.
## Verification note
Items in A–E were derived from the committed parity manifest and SQL, not from a live render; a visual pass over the published template should confirm each before/after (legend rendering, label truncation on long agent/user IDs in top-5 bars, and mobile layout are the main things only a live check can judge).
Contributor guide
Research direction
Choose one independently scoped checkbox, then read spec/chart_manifest.yaml, the SQL contracts, and the relevant source under dashboard/looker_studio/ or docs/. Compare it with the published Looker Studio report and configurator source before making changes. Done means the selected item is implemented or intentionally documented and its before/after behavior is verified in the report or page.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- css, gcp, html, sql, yaml
- Domain
- accessibility, analytics, data-visualization, documentation, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100