googleapis / googleapis/python-aiplatform

Per-request `_force_flush_otel()` in ADK template blocks streaming responses

Aperta
#6,807 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
api: vertex-ai
Lingua principale
Python
Stelle
905
Fork
465
Merge medio
1g 13h
PR unite (30g)
44

Descrizione

### Environment

- `google-cloud-aiplatform` (vertexai SDK)
- `vertexai/agent_engines/templates/adk.py`
- Agent Runtime with `identity_type=AGENT_IDENTITY`
- ADK >= 1.17.0

### Description

#### Problem

The ADK template's `streaming_agent_run_with_events` and `async_stream_query` methods call `_force_flush_otel()` in their `finally` blocks on **every request**. This triggers a synchronous `BatchSpanProcessor.force_flush()` that blocks the response stream until all queued spans are exported to `telemetry.googleapis.com`.

In Agent Identity environments where the export uses certificate-bound tokens (mTLS via SPIFFE/WIF), the authentication overhead is significantly higher than standard OAuth, and the blocking duration becomes noticeable to end users. Setting `GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=false` eliminates the overhead, confirming the flush is the sole cause.

#### Root Cause

`_force_flush_otel()` in `adk.py` calls `tracer_provider.force_flush()` via `asyncio.to_thread()`, which invokes `BatchSpanProcessor._export(BatchExportStrategy.EXPORT_ALL)` — a blocking synchronous export. The default `OTEL_BSP_EXPORT_TIMEOUT` is 30,000ms, so the flush can block for up to 30 seconds depending on the mTLS authentication latency.

The comment on the call site reads:
```
# Avoid telemetry data loss having to do with CPU throttling on instance turndown
```

This concern is valid, but per-request flush is the wrong mechanism for it. `BatchSpanProcessor` already exports asynchronously at a configurable interval (default 5s). Telemetry data loss on shutdown should be handled by a shutdown hook, not by blocking every request.

#### Reproduction

1. Deploy an ADK agent to Agent Runtime with `identity_type=AGENT_IDENTITY`
2. Ensure telemetry is enabled (`GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=true`, or leave at default for ADK >= 1.17)
3. Send a streaming query
4. Observe a significant gap between agent execution completion and response delivery
5. Set `GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=false` and repeat — the gap disappears

#### Expected Behavior

The OTel flush should not block the request path. `BatchSpanProcessor` is designed to export spans asynchronously in background threads. Per-request `force_flush()` negates this design and introduces user-facing latency proportional to the telemetry export duration.

When running the same agent on Cloud Run with ADK `Runner.run_async()` directly (bypassing the ADK template wrapper), no such overhead exists — the agent execution time is the same, but the framework-level gap is negligible.

#### Code References

**Call sites** (per-request flush in `finally` blocks):
- `streaming_agent_run_with_events`
- `async_stream_query`

**Flush implementation**:
- `_force_flush_otel`

### Workaround

Set `GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=false` in the Agent Runtime deployment's `env_vars`. This disables Cloud Trace export but does not affect BigQuery Agent Analytics Plugin (which uses its own data path).

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.