GoogleCloudPlatform / GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK
Feature request: MCP App interactive dashboard (Langfuse/Arize-style) for agent_events
- Dominant language
- Python
- Stars
- 47
- Forks
- 21
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 33
Description
## Summary
Proposal: an **MCP App dashboard** for BigQuery Agent Analytics — an interactive, Langfuse/Arize-style observability dashboard that renders **directly inside MCP hosts** (Claude / Claude Desktop, VS Code Copilot, Goose, etc.) via the [MCP Apps extension](https://modelcontextprotocol.io/extensions/apps/overview), backed by the `agent_events` table that the BigQuery Agent Analytics SDK/plugin already populates.
Today the visualization story for Agent Analytics is the [Looker Agent Analytics block](https://marketplace.looker.com/marketplace/detail/agent_analytics), which is great but requires a Looker instance. Many teams instrumenting agents with this SDK are already living in MCP hosts. An MCP App would let them ask *"show me my agent dashboard"* in Claude and get an interactive dashboard in the conversation — no extra BI deployment, using their existing Application Default Credentials.
## Motivation / prior art
- **Arize dashboards** (https://arize.com/dashboards/): flexible widget building blocks, slice/filter by model/version/dimension, drill-down from any widget into traces.
- **Langfuse custom dashboards** (https://langfuse.com/docs/metrics/features/custom-dashboards): widgets = *metric (count / latency / cost / tokens) × dimension (model, user, trace name) × filters × chart type (line / bar / pie / time-series)*, plus curated prebuilt Latency / Cost / Usage dashboards.
- **Looker Agent Analytics block**: token consumption & cost, user engagement, tool execution volume & failure rates, P50–P99 latency distributions, drill-down popups.
The proposal is essentially: bring this same widget model to MCP hosts, scoped to the Agent Analytics `agent_events` schema.
## Proposed design
### Architecture
A small MCP server (stdio, Node or Python) shipped in this repo (e.g. `apps/mcp-dashboard/`):
1. **Tools**
- `show_dashboard(view, time_range, filters)` — declares `_meta["ui"].resourceUri = "ui://bqaa/dashboard.html"`; the host renders the UI resource in a sandboxed iframe.
- `query_agent_metrics(metric, dimension, time_range, filters, granularity)` — parameterized aggregate queries against `agent_events`; called both by the model (text answers) and by the dashboard iframe via the MCP Apps `tools/call` bridge for interactive refresh/drill-down.
- `get_trace(trace_id, time_range)` — trace reconstruction for drill-down from any widget (mirrors Arize's widget→trace workflow).
2. **UI resource** — a single self-contained HTML bundle (inline JS/CSS, no external CDNs, CSP-friendly) registered at `ui://bqaa/dashboard.html`.
3. **BigQuery access** — `@google-cloud/bigquery` (or `google-cloud-bigquery`) with ADC; configured via env (`BQAA_PROJECT`, `BQAA_DATASET`, `BQAA_TABLE`). All queries use parameterized `@start`/`@end` timestamp predicates so the partitioned table is never full-scanned, plus a `maximumBytesBilled` guard.
### Prebuilt views (tiles map 1:1 to `agent_events` queries)
| View | Tiles | Source |
|---|---|---|
| **Overview** | events, sessions, users, agents, error-rate, p95 latency stat tiles; events-over-time; error-rate trend | survey aggregates over the full table |
| **Latency** | p50/p95/p99 total + TTFT by agent & model; latency-over-time; generation vs TTFT split | `LLM_RESPONSE` → `latency_ms.$.total_ms`, `$.time_to_first_token_ms` |
| **Tokens & cost** | prompt/completion tokens over time by model; top sessions by tokens; per-model averages (cost = tokens × user-supplied price table, following the SDK's convention of not hardcoding prices) | `attributes.$.usage_metadata.*` |
| **Tools** | call volume, failure rate, p95 latency per tool/origin; failure trend | `TOOL_COMPLETED` |
| **HITL** | requests vs completions, avg/max wait per request type | `HITL_*` pairs |
| **Agents** | delegation graph (parent→child span self-join), per-agent volume/error split | `parent_span_id` self-join |
### Interactivity (the part text answers can't do)
- Global time-range + agent/model/user filter bar; every widget re-queries through the iframe→host `tools/call` bridge.
- Click any bar/point → drill down to matching sessions/traces → `get_trace` renders the event timeline inline.
- "Custom widget" builder: pick measure × dimension × chart type (Langfuse-style), so users aren't limited to the curated views.
### Security / cost guardrails
- Read-only: only `SELECT` against the configured table; queries are parameterized (no SQL injection via the iframe bridge).
- Sandboxed iframe per MCP Apps spec; the UI can only reach BigQuery through the server's typed tools.
- Partition pruning enforced (required time predicate), `maximumBytesBilled` cap, and dry-run cost surfacing in the UI footer.
## Questions for maintainers
1. Would you accept this in-repo (e.g. `apps/mcp-dashboard/`), or is a separate community repo preferred?
2. Preference for Node vs Python for the MCP server? (MCP Apps examples/SDK are TypeScript-first.)
3. Any interest in sharing the SQL layer with the Looker block's derived-table definitions so the two stay consistent?
I have a working prototype (server + Overview/Latency/Tokens/Tools views) I'm happy to contribute — will link the branch/PR here.
Contributor guide
Research direction
Start by reviewing the proposed apps/mcp-dashboard/ entry point, the existing agent_events schema, and the Looker Agent Analytics block for reusable query definitions. Resolve the repository, runtime, and SQL-layer questions before implementing the MCP server, dashboard views, parameterized metric queries, and trace drill-down described here; done requires maintainer agreement and working coverage for the proposed views.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, node.js, python
- Domain
- backend, data, frontend, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100