googleapis / googleapis/python-aiplatform

Per-request `_force_flush_otel()` in ADK template blocks streaming responses

オープン
#6,807 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
api: vertex-ai
主要言語
Python
スター
905
フォーク
465
平均マージ
1日 13時間
マージ済み PR(30日)
44

説明

### Environment

- `google-cloud-aiplatform` (vertexai SDK)
- `vertexai/agent_engines/templates/adk.py`
- Agent Runtime with `identity_type=AGENT_IDENTITY`
- ADK >= 1.17.0

### Description

#### Problem

The ADK template's `streaming_agent_run_with_events` and `async_stream_query` methods call `_force_flush_otel()` in their `finally` blocks on **every request**. This triggers a synchronous `BatchSpanProcessor.force_flush()` that blocks the response stream until all queued spans are exported to `telemetry.googleapis.com`.

In Agent Identity environments where the export uses certificate-bound tokens (mTLS via SPIFFE/WIF), the authentication overhead is significantly higher than standard OAuth, and the blocking duration becomes noticeable to end users. Setting `GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=false` eliminates the overhead, confirming the flush is the sole cause.

#### Root Cause

`_force_flush_otel()` in `adk.py` calls `tracer_provider.force_flush()` via `asyncio.to_thread()`, which invokes `BatchSpanProcessor._export(BatchExportStrategy.EXPORT_ALL)` — a blocking synchronous export. The default `OTEL_BSP_EXPORT_TIMEOUT` is 30,000ms, so the flush can block for up to 30 seconds depending on the mTLS authentication latency.

The comment on the call site reads:
```
# Avoid telemetry data loss having to do with CPU throttling on instance turndown
```

This concern is valid, but per-request flush is the wrong mechanism for it. `BatchSpanProcessor` already exports asynchronously at a configurable interval (default 5s). Telemetry data loss on shutdown should be handled by a shutdown hook, not by blocking every request.

#### Reproduction

1. Deploy an ADK agent to Agent Runtime with `identity_type=AGENT_IDENTITY`
2. Ensure telemetry is enabled (`GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=true`, or leave at default for ADK >= 1.17)
3. Send a streaming query
4. Observe a significant gap between agent execution completion and response delivery
5. Set `GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=false` and repeat — the gap disappears

#### Expected Behavior

The OTel flush should not block the request path. `BatchSpanProcessor` is designed to export spans asynchronously in background threads. Per-request `force_flush()` negates this design and introduces user-facing latency proportional to the telemetry export duration.

When running the same agent on Cloud Run with ADK `Runner.run_async()` directly (bypassing the ADK template wrapper), no such overhead exists — the agent execution time is the same, but the framework-level gap is negligible.

#### Code References

**Call sites** (per-request flush in `finally` blocks):
- `streaming_agent_run_with_events`
- `async_stream_query`

**Flush implementation**:
- `_force_flush_otel`

### Workaround

Set `GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=false` in the Agent Runtime deployment's `env_vars`. This disables Cloud Trace export but does not affect BigQuery Agent Analytics Plugin (which uses its own data path).

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。