BigQuery Agent Analytics: custom_tags / custom_labels Path Mismatch

# BigQuery Agent Analytics: custom_tags / custom_labels Path Mismatch

## Summary

There is a JSON path mismatch between how the ADK plugin **writes** custom
metadata to BigQuery and how the SDK **queries** it. Data written via
`BigQueryLoggerConfig.custom_tags` is stored at `attributes.custom_tags.*`,
but `TraceFilter.custom_labels` queries `attributes.labels.*`. The two
paths never intersect, making custom tags effectively unqueryable through
the SDK's filtering API.

## Versions

- `bigquery-agent-analytics`: 0.2.3
- `google-adk`: 1.32.0

## Detailed Analysis

### Write Path (ADK Plugin)

**File**: `google/adk/plugins/bigquery_agent_analytics_plugin.py`

The `BigQueryLoggerConfig` dataclass (line 593) has a `custom_tags` field:

```python
class BigQueryLoggerConfig:
custom_tags: dict[str, Any] = field(default_factory=dict)
```

In `_enrich_attributes()` (lines 2817-2818), these tags are written to
the `attributes` JSON column at the `custom_tags` key:

```python
if self.config.custom_tags:
attrs["custom_tags"] = self.config.custom_tags
```

**Result**: setting `BigQueryLoggerConfig(custom_tags={"version": "v0.5"})`
produces this JSON in the `attributes` column:

```json
{"custom_tags": {"version": "v0.5"}, ...}
```

### Read Path (SDK TraceFilter)

**File**: `bigquery_agent_analytics/trace.py`

The `TraceFilter` dataclass (line 436) has a `custom_labels` field:

```python
custom_labels: Optional[dict[str, str]] = None
```

In `to_sql_conditions()` (lines 603-613), this generates a query against
the `labels` key — **not** `custom_tags`:

```python
if self.custom_labels:
for i, (key, value) in enumerate(self.custom_labels.items()):
param_key = f"label_key_{i}"
param_val = f"label_val_{i}"
conditions.append(
f"JSON_VALUE(attributes,"
f" CONCAT('$.labels.', @{param_key}))"
f" = @{param_val}"
)
params.append(bigquery.ScalarQueryParameter(param_key, "STRING", key))
params.append(bigquery.ScalarQueryParameter(param_val, "STRING", value))
```

**Result**: querying `TraceFilter(custom_labels={"version": "v0.5"})` generates:

```sql
WHERE JSON_VALUE(attributes, '$.labels.version') = 'v0.5'
```

This looks at `attributes.labels.version`, but the data lives at
`attributes.custom_tags.version`. **The query returns zero rows.**

### What Does Populate `attributes.labels`?

The `labels` path is populated from `llm_request.config.labels` — a
per-LLM-call mechanism, not a static plugin config:

**File**: `google/adk/plugins/bigquery_agent_analytics_plugin.py` (lines 3243-3244):

```python
if labels := getattr(llm_request.config, "labels", None):
attributes["labels"] = labels
```

The ADK framework automatically sets one label — the agent name:

**File**: `google/adk/flows/llm_flows/base_llm_flow.py` (lines 65, 1182-1189):

```python
_ADK_AGENT_NAME_LABEL_KEY = 'adk_agent_name'

llm_request.config.labels = llm_request.config.labels or {}
if _ADK_AGENT_NAME_LABEL_KEY not in llm_request.config.labels:
llm_request.config.labels[_ADK_AGENT_NAME_LABEL_KEY] = (
invocation_context.agent.name
)
```

So `attributes.labels` currently contains only `{"adk_agent_name": ""}`,
set automatically by the framework. Users have no documented way to add
custom entries here.

## Impact

This blocks the primary use case for custom tags: **version-aware session
filtering**. We need to tag every BQ event with the deployed software
version so that quality reports and evolution pipelines can filter sessions
by version (e.g., "give me all sessions from agent version v0.5").

Without this, the evolution pipeline cannot distinguish sessions from
different skill versions, causing it to analyze a mix of pre- and
post-evolution traces — poisoning the signal.

### Workaround

Query `$.custom_tags.*` directly with raw SQL instead of using
`TraceFilter.custom_labels`:

```python
# Manual workaround — bypasses TraceFilter
query = """
SELECT *
FROM `{project}.{dataset}.{table}`
WHERE JSON_VALUE(attributes, '$.custom_tags.version') = @version
"""
```

This works but loses all the conveniences of `TraceFilter` (composable
filters, parameterized queries, pagination, integration with
`client.list_traces()` and `client.evaluate_categorical()`).

## Additional Issues

### 1. `custom_labels` not exposed in `from_cli_args()`

`TraceFilter.from_cli_args()` (lines 444-489) does not accept a
`custom_labels` parameter:

```python
@classmethod
def from_cli_args(
cls,
last: str | None = None,
agent_id: str | None = None,
session_id: str | None = None,
user_id: str | None = None,
has_error: bool | None = None,
limit: int = 100,
) -> "TraceFilter":
```

Even if the path mismatch were fixed, CLI-based workflows (quality report
scripts, evolution agent runners) cannot pass custom labels.

### 2. Naming inconsistency

- Plugin config: `custom_tags` (dict[str, Any])
- TraceFilter: `custom_labels` (dict[str, str])
- BQ write path: `attributes["custom_tags"]`
- BQ read path: `attributes.labels.*`
- ADK framework: `llm_request.config.labels`

Four different names for conceptually the same thing.

### 3. No documentation

`TraceFilter.custom_labels` has no docstring or mention in the class-level
documentation (lines 422-426):

```python
"""Filtering criteria for listing traces.

All fields are optional. When multiple fields are set they
are combined with AND logic.
"""
```

## Proposed Fix

### Option A: Align write path to read path (preferred)

Change `_enrich_attributes()` to write `custom_tags` to the `labels`
key so that `TraceFilter.custom_labels` can find them:

```python
# bigquery_agent_analytics_plugin.py, line 2817-2818
if self.config.custom_tags:
attrs.setdefault("labels", {})
attrs["labels"].update(self.config.custom_tags)
```

This merges user-defined tags with ADK-set labels (like `adk_agent_name`)
under one path, making everything queryable via `TraceFilter.custom_labels`.

**Migration**: existing data at `$.custom_tags.*` would need a backfill
query or a dual-read approach during transition.

### Option B: Align read path to write path

Change `TraceFilter.to_sql_conditions()` to query `$.custom_tags.*`:

```python
# trace.py, line 607
conditions.append(
f"JSON_VALUE(attributes,"
f" CONCAT('$.custom_tags.', @{param_key}))"
f" = @{param_val}"
)
```

Simpler, no migration needed, but diverges from the ADK's `labels`
convention.

### Option C: Support both paths

Query both `$.labels.*` and `$.custom_tags.*`:

```python
conditions.append(
f"(JSON_VALUE(attributes, CONCAT('$.labels.', @{param_key})) = @{param_val}"
f" OR JSON_VALUE(attributes, CONCAT('$.custom_tags.', @{param_key})) = @{param_val})"
)
```

Most flexible but adds query complexity.

### Additional fixes needed

Regardless of which option:

1. Add `custom_labels` parameter to `TraceFilter.from_cli_args()`
2. Add documentation to `TraceFilter` explaining `custom_labels`
3. Unify naming: pick either "tags" or "labels" and use it consistently

## Reproduction Steps

```python
from google.adk.plugins import BigQueryAgentAnalyticsPlugin
from google.adk.plugins.bigquery_agent_analytics_plugin import BigQueryLoggerConfig
from bigquery_agent_analytics import Client, TraceFilter

# 1. Log events with custom_tags
config = BigQueryLoggerConfig(
custom_tags={"experiment": "v2", "version": "0.5"}
)
plugin = BigQueryAgentAnalyticsPlugin(
project_id="my-project",
dataset_id="agent_logs",
table_id="events",
config=config,
)
# ... run agent, events are logged ...

# 2. Try to query using custom_labels
client = Client(project_id="my-project", dataset_id="agent_logs")
traces = client.list_traces(
TraceFilter(custom_labels={"experiment": "v2"})
)
# Returns EMPTY — data is at $.custom_tags.experiment,
# but query looks at $.labels.experiment
```

## Use Case: Version-Aware Evolution Pipeline

The specific use case driving this report:

```
Deploy agent v0.5 (with custom_tags={"agent_version": "v0.5"})
→ Sessions logged to BQ with version tag
→ Quality agent runs daily, filters by version, creates issues
→ Evolution agent runs weekly:
1. Query BQ: all sessions where agent_version = "v0.5"
2. Run analyst fleet on failures
3. Produce SKILL.md v0.6
4. Verify open issues are fixed
5. Create PR
```

This requires `TraceFilter(custom_labels={"agent_version": "v0.5"})` to
work end-to-end, which it currently does not.