googleapis / googleapis/google-cloud-java

[java-storage] Excessive ALTS handshake logs from gke-metadata-server when using Java GCS client with gRPC

オープン
#12,633 コメント 4 件 リアクション 0 件 担当者 0 名 GitHub で見る
api: storage type: question
主要言語
Java
スター
2.1k
フォーク
1.2k
平均マージ
1日 23時間
マージ済み PR(30日)
154

説明

### Description

We are observing an extremely high volume of informational logs (approximately 500,000 to 1,000,000 per day) from the `gke-metadata-server` container on our GKE cluster.

This appears to be triggered by our Java application that uses the `google-cloud-storage` library with the gRPC transport option to interact with Google Cloud Storage. Our application handles a high volume of requests, writing to GCS approximately **1 million times per day, with peaks of up to 1,000 writes per second**.

The excessive logging is causing a significant increase in costs for our monitoring and logging tool (Datadog).

The most frequent log messages are related to ALTS session handshakes:

```
[rpc-id:b0b2453b42e5f6b2] Closing ALTS session from pod "ml-test-common-historical-1".
[rpc-id:b0b2453b42e5f6b2] Use node identity as channel identity to connect to service: "storage.googleapis.com"
[rpc-id:b0b2453b42e5f6b2] Starting ALTS session from pod "ml-test-common-historical-1" running as test-common/ml-test-common-app to act as ml-test-common@evaluation-mila.iam.gserviceaccount.com
[rpc-id:b0b2453b42e5f6b2] Processing ALTS handshake request from "10.80.12.18:53498".
```

Given the high traffic, we suspect that the client is creating new ALTS sessions too frequently, rather than reusing existing ones, which results in this verbose logging.

### Environment

* **`google-cloud-storage` version:** `2.55.0`
* **Transport:** gRPC (`StorageOptions.grpc()`)
* **Environment:** Google Kubernetes Engine (GKE)
* **GKE Cluster Version:** `1.33.4-gke.1036000`
* **Workload Identity Enabled:** Yes
* **Java Version:** `24`

### Code Snippet

Our `Storage` client is initialized as a singleton and reused throughout the application's lifecycle. The following code demonstrates our usage pattern.

```java
// This class is a singleton
class MyClass {
final Storage storage;

public MyClass() {
// Client is initialized once using gRPC transport
this.storage = StorageOptions.grpc()
.setProjectId("your-gcp-project-id")
.build()
.getService();
}

@Override
public void saveData(String dataId, ByteBuffer data) throws Exception {
BlobInfo blobInfo = toBlobInfo(dataId);
Blob blob = this.storage.create(blobInfo, BlobTargetOption.doesNotExist());
try (WriteChannel wc = blob.writer()){
wc.write(data);
}
}
}
```

### Investigation So Far

We opened a support case with the Google Kubernetes Engine team. Their recommendation was to **"adjust the client library's connection pool to prevent frequent session creation."**

However, it is not clear from the `google-cloud-storage` or `google-api-grpc` documentation how to configure or tune the underlying gRPC connection pool for this purpose.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。